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ABSTRACT 

Project CREATE was a 4-year federally funded research 
program which sought to determine ways in which computer technology 
could improve the education of learning-disabled students. Early work 
on the project centered on the development of a theoretical and 
conceptual foundation? a hierarchical conceptual model was 
constructed that related school performance in reading to underlying 
cognitive processing skills, perceptual enabling skills r and 
neuromuscular vision skills* Two California school districts were 
involved in the project: the Fremont Union High School District and 
the Cupertino Union School District. A software selection and 
evaluation tool (included in Appendix B) was developed, several 
adaptive software handbooks w-re produced, and two software programs 
were created (TurboScan and V: ^rdsworth) . Elementary students who 
received the TurlxjScan software treatment improved in_ their ability 
to rapidly discriminate critical features assocjated with character 
and word recognition, and demonstrated better reading skills. 
Secondary students who received the Wordsworth software treatment for 
cognitive processing development improved in their ability to 
recognize literal facts and details in narrative text and, to some 
extent, to make inferences. Student-machine interface conditions that 
acted as barriers to effective study were identified, and a 
facilitating interface was designed. An appendix contains scope and 
sequence information., ( Author/ JDD) 
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Abstract 



The CREATE Project was a multi-year study of ways in which computer 
technology might be used to better serve the learning disabled. Early work 
centered on the development of a theoretical and conceptual foundation for 
subsequent investigation based on a comprehensive review of the literature 
on leamrag, cognition, perception, software design, selection and utilization, 
and computer experimentation in education. A number of occasional papers 
were issued to summarize the results of these investigations. 

A heirarchical conceptual model was constructed that related school 
performance in reading to tmderlying cognitive processing skills, perceptual 
enabling skills, and neuromuscular vision skills, kesearch was conducted in 
CREATE to explore the potential of the computer for improving reading 
through special purpose software tools intended to improve students* 
cognitive processing and perceptual enabling skills. (Researdi was conducted 
outside the project in a related study on neuromuscular skills.)' Two 
cooperating school districts (one elementary and one secondary) participated 
in the CREATE studies as did other volxmteer schools aroimd the coimtry. 

A new software selection and evaluation tool was developed, field tested in 
volunteer schools, and revised in line with suggestions received. This form 
focuses the attention of school staff on the educational applications of 
software consistent with the model described above. 

Several adaptive software handbooks were developed, and two published 
commercially, to demonstrate tl\e ways in which computer study (using 
specific commercial software) can be better integrated into the curriculum in 
order to benefit the handicapped. 

Two new software programs were developed for the experimental research 
conducted in CREATE. With these tools we were able to show that significant 
gains .can be made by learning disabled students in relatively short periods of 
time on the underlying skills associated with reading. 

Students who received the TurboScan software treatment (for perceptual 
enabling skill development) in the cooperating schools improved in their 
ability to rapidly discriminate critical features associated with character and 
word recognition. Students at the elementary level demonstrated better in 
their reading skills as well. There was no clear evidence favoring specific 
design featiu^es being present or absent in the specially developed TurboScan 
software. These features included immediate loiowledge of progress, 
corrective feedback, relative difficulty of exit criterion, and leamer-controUed 
rate of presentation. 
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Secondary students who .received the Wordsworth software treatment (for 
cognitive processing development) improved in their ability to recognize 
literal facts and details in narrative text and, to some extent, make inferences, 
contributing to better comprehension- An important aspect of the 
Wordsworth research program involved the identification of student - 
machine interface conditions that acted as barriers to effective study and the 
red&ign of the interface to facilitate computer study by the learning disabled. 
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Overview of the CREATE Project 



The Office of Special Education Programs (OSEP) in the Department of. 
Education awarded four research grants in October of 1983 to undertake 
systematic research on the ways in wMch technology, particularly computer 
technology, could be used to better serve in the education of handicapped 
students. 

In the competition, the American In^>titutes for Research (AIR) was selected to 
imdertake a program of research outlined in the proposal, a four-year, 
multipurpose investigation. Perhaps the study could best be described as 
proactive. That is, rather than merely assessing the extent to which 
computers were currently being used In the schools and trying to ascertain the 
benefits and problems that might be attributable to their use, AIR attempted 
to explore the unrealized potential of computer technology through a series 
of investigations that \rauld build upon a conceptual framework of theory 
and research to "stretch the limits" of the technology beyond current practice. 

In carrying out this research program^ AIR worked closely with two school 
districts whose administrators and teachers had established themselves as 
leaders in using computers in the schools and whose special education staffs 
were known to be innovative. They provided valuable perspectives during 
the planning of tlie different investigations, helped in deve oping some of 
the innovative computer materials that were prepared, and provided 
"laboratory" settings in which pilot testing and experimentation could take 
place. The school districts were the Fremont Union High School District and 
the Cupertino Union School District, both in California, respectively 
representing grades 9-12 and K-8. 

The domains of interest in the original proposal, to be modified as a result of 
initial year investigations, were 

1. Building learners* enabling skills, processing skills, and performance 
skills. 

2. Stimulus-properties and user-friendly tchniques to incorporate in 
commercial and teacher-authored materials 

3. Methods for evaluating, selecting, and using existing materials for 
suitability to handicapped learners 

4. Logistics of hardware, software, arid training of staff and student 
leadership 
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5. Innovative subject matter applications in mainstream,, resource 
room, home, and community-based contexts 

6. Networks, ir*^eractive systems, and the inter-school pooling of 
resources, information, and instructional content 



One of the characteristics of multiple year grants is that they afford the 
researchers an opportimity armually to shift the emphasis toward 
particularly promising aspects of the originally targeted purposes. Such 
was the case in this study. Because the study was one of four that were 
concurrently funded by OSEP, a review of their proposals made it clear 
that issues relating to classroom logistics and utilization of technology 
(#4, 5, and 6 above) would be amply covered by those projects, making 
it desirable for AIR to pay proportionately more attention to issues #1, 
2, and 3 which were uot as thoroughly covered, 

Midway through the course of the pro^t, staff became more focused 
on two research questions that influenced our experimentation in the 
third ai.d fourth years. Put simply, they were: 

1. Can miaocomputers/pardcular software be used to teach/improve 
the learning performance of learning disabled students with particular 
characteristics? 

Z What feature? are critical in software for learning disabled students 
with particular characteristics? 



Theoreticai and Conceptual Foxmdations 
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Theoretical anu Conceptual Foundations 



The overaU theoretical framework that was developed during the first year of 
the CREATE study suggested that a number of learning disabled students, 
whose reading performance in academic subject matter was not 
commensurate with, their general abiUty, might lack certain cognitive 
processing skills which fadUtate comprehension. Likewise, a number of poor 
readers mi^t lack lower order perceptual enabling skiUs, such as the 
"automatic" recognition of words based on their length, shape and initial 
lettei-(s), dut are a. precondition to efficient cognitive processing. The general 
research question posed was whether computer technology could be used to 
address these skills and ttius enhance poor readers' ability to function at a 
higher level. 

Accordingly, experiments were designed and special purpose software 
developed to focus first.on research relating to perceptual enabling skills 
(using a newly developed software research tool, TurboScan) and later on 
research relating to cognitive processing (a second newly developed software 
research tool, " Vordsworth) . 

Figure 1 shows the conceptualized heirarchy of skills at the perceptual- 
enabling and cognitive-processing levels. In examining Figure 1, it should be 
pointed out that 

• 

• Neuromuscular vision skills were investigated outside project 
CREATE in a concurrent study. This project, led by Dr. David Grisham, 
an AIR researcher who held a joint appointment at the School of 
Optometry, University of California, Berkeley, found that computer 
technology could be used to facilitate vision training in binocular 
fusion and near/far accommodation. 

^•♦Most , but riot all, of the scope and sequence topics outlined in 
the perceptual enabling skills level were covered in the TurbQgcan 
software. (See Appendbc A for the Scope and Sequence chart.) 

* A relatively small proportion of the full range of cognitive prpcagging 
skills were addressed in the Wordsworth software. Specifically, we 
focussed on the lowest level of comprehension skills - identifying 
facts and details in narrative text (See Appendix A.) 

•No attempt was made to study the arademic performance of students* 
reading in the subject areas (e.g., EngUsh and mathematics) inasmuch 
. as the sister research projects elsewhere were studying patterns of use 
and academic outcomes of classroom use of computers. 



A Conceptual Helrarchy of Skills 
Contributing to Effective Reading 



Academic Performance Level; 
E(f active, efficient reading 
in scliool contexts 

Skiils 

Oral and silant reading 
Vocabulary mastery 
Spelling, accuracy 



Indicos 



Knowledgs acquisition 
Language skills 
Test taking 



Indices 



Cognitive Processing Level: 
Rapid, accurate, acquisition 
and comprehension of .text 

Skills 

Note facts and details Organized relationships 

Follow sequence of events Comprehension 
V Draw inferences " ^ ^, 

\Grasp main ideas Problem^solvmg 

Perceptual Enabling Level: 
Rapid, accurate, visuai discrimination 
of symbols and words 




Skills 

Perceive character feattires 
Perceive sets of characters 
Identify words by '©nQ^* shape, 
and letter combinations 



; identiTv 
V and lei 



Indices 

Differentiate letter shapes 
Recognize tetter combinations 
Recognize words on sight 
Identify target words in a field 



Neuromuscular Vision Level: 
Efficient binocular vision 
at near and far distances 




Skills 

Muscle balance 
Depth perception 
Accommodation 
Sensory and motor fusion 
Ocular motility 



Indices 



Phoria and eye alignment 
Storopsis discrimination 
Adjusting focus far and noar 
Fusion speed and endurance 
Tracking ey© movements 



Figure 1. A conceptual heirarchy of skills contributing to effective 



Occasional Papers 



Early in the CREATE project a number, of reports werfi prepared to 
summarize the "state of the art" (in 193^1: with respect to the application of 
computer technology to learning. Tne first two papsxs dted below were 
central to the formulation of later CREATE experiments. The occasional 
papers included 

• Weisgerber, R. A., (198-1) Implications of Research and Theory 
for the Use of Computers w'th the Learning Disabled 

• Rubin, D., Blake, P., and Aten-Iossi, B. (1984) The Learning Disabled 
and Computer-Based Education: Program Design. Strategies 

• Bakke, T., (1984) Bdsting and Emerging Technologies in Education: 
A Descriptive Overview 

• Weisgerber, R. A. and Blake, P.L., (1984) The Evaluation and 
Selection of Instructional Software for Use with ti\e Learning Disabled 

• Bakke, T., (1984) Optimal Approaches to Miaocomputer 
Implementation in the Schools 

• Rossi, R., Rubin, D., and Bakke, T., (1984) Promoting Use of 
Technology in tiie Schools: Ai\ Organizational Analysis of 
California's Teacher Education and Computer Centers 

One additional paper will be released concurrently witii tiiis final report It 
will address "breakthroughs" that have been observed and reported (largely 
anecdotally) when disabled persons have been given the opportunity to use 
computers. An example of a breakthrougii is ti\e case of a dyslexic young man 
in Oregon whose expressive abilities were severely limited but who, tiurough 
use of the, computer, now has become a uiuversity teacher in visual design. 
Anecdotes about changed life ©qjeriences havo served as beacons pointing 
out new directions for researchers, practitioners, and tiie disabled. 



Dissemination 

The first and second of these papers have been widely circulated to requesting 
educational and research institutions. The fourth paper was shared with _ _ 
schools that had yolunteered to participate in field testing of tiie evaluation 
and selection form and other interested schools. The third, fiftii, and sbcth 
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papers were narrower in scope and were circulated principally to requesting 
schools from withir. California. 

Copies of all occasional papers were submitted to the Office of Special 
Educatioii Programs and to the three other projects concurrently funded to 
investigate the application of technology to the education cf the handicapped: 
University of California (Santa Barbara), the University of Maryland, and 
Vanderbilt University. 

Several of the occasional pap' s were specifically adapted for journal 
pubEcation or referenced hi youmals. These included: 

The Learnin g Disabled and Computer Based Education; Pro-am 
Design: Strategies, published by the International Council for 
Computers in Sducatibn in theL' SIG Bulletin, October/November,/ 
December, 1985. 

Implication, of Research and Theory for the Use of Computers with the 
Learning Disabled referenced by The Computing Tead\er, May, 1986. 

Ai'iides that described or dted various aspects of ATR's research in the 
CREAiE. project induded: 

Weisgerber,. R. and Rubin, D. Designing and Using Software for the 
Learning Disabled. In Toumal of Reading , Writin g and Learning 
Disabilities, Vol. 1, No. 2, Winter, 1985, pp. 133-138. 

Rubin, D. and Weisgerber, R. The; Center for.Research and Evaluation 
in the Application of Technology to Education. In T.H. E. Toumal. 
February, 1985, pp. 83-87. 

Rudy, R. Effective reading goal of school's computer studies. 
Newspaper story in the Times-Tribu ne, published in Palo Alto, 
Sunday, Jvaie 22, 1986. 

Rosenfeld, N. New Vistas for Disabled: Computers Lead the Way. In 
n CompuHng. For Apple Users. August/September 1986. 

New Computer Study Funded. ACLD Newsbriefs. March/April 

1984, pp. 1,13. 

. Adapting Technology to the Needs of Handicapped Learners. 

Entre Nous. May 1984, pp. 1-2. 

. Project CREATE. REACH Bulletin. Issue 1: March, 1984; Issue 2: 

October, 1984; Issue 3: November, 1985; Issue 4: November, 1986. 
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Enhancing the Selection and Use of Available Software and 
Disseminating Information about Computer Use 



Soft\\'are Selection 

The process by which educational software is selected by school districts varies 
widely. One product of our research in 1985 was a new technique for 
evaluating and selecting progran\s designed to 

(a) focus the attention of the educator-reviewer on issues related to 
educational values in tiie software while providing for clerical input 
on information that is simply descriptive of the product, 

(b) inti-oduce a distinction between software selected for academic 
purposes (performance skills) and software selected to meet specific 
deficits of disabled youth (enabling skills involving perception; 
processing skills involving cognition) 

(c) provide a way of rating and weighting the merite of the software 
according to instinctional design elements (instructional strategy, 
learner cbnbrol, and feedback/reinforcement) and the quality of 
individual components (doomientation, supplementary materials, 
and program operation and presentation). 

The Software Evaluation and Selection Form included in Appendix B of this 
report resulted from a thorough search and analysis of published selection 
forms available at that time. and an iterative process of development and 
evaluation with school personnel in volunteer "field test" school districts 
around tiie country. 

It is worthy of note that the evaluation and selection procedure as shown in 
Appendix B would lend itself very well to a computerized data base retiieval 
system ti\a«- could er^able teachers in a district to access relevant software 
according 'J its educational purpose rathe*- tiian by tide or otiier general 
terms. While such a task cculU uot be accomplished witiiin tite framework o 
CREATE research, this remains an objective worthy of support by OSEP. 



Integrating Computer Software into ti\e Curricalum 

Frequentiy, criticism has been directed toward popular computer software as 
being interesting but not well correlated with the core educational 
curricalum. Typical of the interesting but seemmgly unrelated software are 
some of tire more challenging programs that reqtiire cognitive "problem 
solving" skills. Accordingly, some software tiiat has been very popular in 
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terms of unit sales has not been integrated into the curriculum but rather 
tends to be used as a separate "activity," and treated ar a reward. When 
computers are used in an unrelated way, the student can hardly be expected to 
transfer knowledge gained through computer study (if any) to the basic 
educational goals in the core curricula. 

Research has shown that the most common use of the computet in the 
schools is for drill and practice. Drill and practice can be a relatively dull 
application of technology when it simply means the workbook-like 
presentation of math problen\s, vocabulary, spelling word? and so forth. 
Consequendy, it is not surprising that another criddsm leveled at some 
software is that it is unimaginadve and repetitive. On the other hand, some 
popular commercial software, such as Reader Rabbit, has motivational 
characteristics but some teachers may not know how to tie it in with the 
airriculum as effectively as they might. 

A fundamental axiom of good instructional planning in a computer-using 
classroom is that computer study and classroom study should be tied together 
ia appropriate, mutually reinforcng ways. To demonstrate the feasibility of 
this premise, we selected five popular (in the schools) educational software 
products, and designed and prepared adaptive teacner guides showing how 
the software could complement regular class activities. The Fa(;tQry, Reader 
Rabbit Kid Writer. Word Spinner, and Story Machine were the commercial 
software products selected to demonstrate that better integration of the 
software would better facilitate the efforts of regtUar education teachers in 
supportmg their disabled students in the basic subject areas. 

The selection of the software and the design of the handboolcs were the joint 
responsibility of the American Institutes for Research project staff and special 
education teachers in the two cooperatinc "laboratory" school districts in 
California, Fremont and Cupertino, respectively representing the secondary 
and elementary grades. Educator and researcher teams were formed to 
develop adaptive handbooks for each software program. 

The strategy followed for the development of the adaptive handbooks v/as to 

• Identify the "educationally relevant" content and/or principles that 
were in the software. 

• Segment the software content into logical inaements or "units" to 
be presented in the v/orkbooks, gradually inaeasing these tmits in 
complexity and difficulty. 

• Prepare a jet of three lessons for each unit of study that would 
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(1) take place in the regular classroom, setting up the 
computer experience that wotild follow, 



(2) be executed in a particular sequence on the computer in 
the lab or other setting in which computer use was scheduled, 

(3) provide follow-up in the regular classroom and "test" the 
transfer of the computer-gained knowledge to subject matter 
being taught there. 

Teacher participation was zniportant in the design, developmeni and 
classroom testing of tiiese adaptive workbooks and many hours were spent b^ 
them outside of school hours in developing the draft materials. Draft 
materials were developed and tested in the schools for Yhe Factory. Reader 
Rabbit Kid Writer, Word Spinner, and Story Machine. The AIR report to 
OSEP entitied CREATE; Software Handbook (September 1983- October 1984) 
included a complete set of the five adapted workbooks. 



Publication of the Workbooks 

The workbooks for two of the software tides mentioned above (The Factory 
and Reader Rabbit) were carried through the complete cycle of preparatic^. 
testing in the schools, and revision. They were subseqiiendy published by the 
firms who sell the software, Sunb\irst and The Learning Company." As 
expected, the commercial availability of these two workbooks, which were 
specifically designed to bridge the gap between the classroom and computer 
study, has extended the usefulness of the software and has been welcomed by 
the hundreds of school districts that have purchased the workbooks. 

Evidence of the way in which these workbooks have been received was 
offered in the publication Closing the Gap. April-May 1986, which regularly 
informs the educational field about advances in computer use in the schools. 

Learning Disabilities teaching materials are now available 
for Sunburst's The Factory and xvill soon be available for 
The Learning Company's Reader Rabbit and Word Spinner. 
These materials were developed by the American Instititues for 
Research, tmder a grant from the Office of Special Education. 

Ive seen The Factory workbook. It contains eight excellent 
modules, each containing pre-computer, computer, and post- 
computer/assessment activities. Many reproducible worksheets 
are included. These are the kinds of materials LD teachers have 
needed for a long time. 

Miriam Fiirst, Contributing Editor 
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Disseminating information about computer u^e 

Throughout the CREATE Project, professional staff made presentations at 
p various conventions, conferences and meetings. These presentations had 

I several purposes: (a) to inform the field about the grant and the CREATE 

project, G)) to inform the field about the strategies for the use of computer 
technology that we were designing and testing, and (c) to offer suggestions to 
the field regarding future directions for software design that would benefit the 
handicapped and related research that needs to be conducted- 

I The following is a list of the appearances and presentations made* 

I R. Weisgerber and D. Rubin. CEC/CASE National Conference and 

Training Workshops on Technology in Special Education. Reno, 
I January, 1984. 

D. Rubin and R. Weisgerber. The CREATE Project: New Directions for 
j New Technologies. California State Federation/CEC 34th Annual 

1 conference, Oakland, November, 1984. 

j D. Rubin and R Weisgerber. Computers and Reading/Learning 

Difficulties. Fourtii Acu^ual Western States Conference, Sail Francisco, 
January, 1985. 

D. Rubin and R Weisgerber. Research in Learning Disabilities: I ew 
Con^puter Approaches to the Treatment of Learning Disabilities. 
ACLD International Conference, San Francisco, February^ 1985. 

D.Rubin. CREATE: Computer Learning Strategies for the Learning 
Disabled. Coimdl ior Exceptional Children, 63rd Annual Convention, 
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An Overview of CREATE Experimentation 



The various experimental studies that ./ere undertaken with the cooperation 
of the laboratory schools were essentially exploratory in nature In a very real 
sense, they were limited "one shot" trials to assess the possible impact of. 
special purpose software on disabled students who had one conimon 
denominator- that they had been classified by the schcK. !s as learning 
disabled. Because reading is a serious problem for many (but not all) learning 
disabled children we focused on reading related experimentation. Thus in 
the broadest sense we were posing ttie general hypothesis that 

The use of special purpose computer sof^vvare can improve 
the reading .skills of learning disabled children. 

At the specific level, we were concerned with trying to demonstrate that 
special purpose computer software could be helpful in developing the 
underlying skills that must be in place if better reading skills are to be applied 
to the variety of reading tasks in school. 

Following the conceptual model outlined in an earlier section of this report, 
we hypothesized a set of relationships that ideally should be solidly in place 
in order for reading to be efficent and effective. Tnak is, we argued that if 
academic performance skills such as reading, vocabulary, and spelling are not 
demonstrated at a level consistent with maturity and general abilit^v the gap 
may be due (in part) to a fundamental deficiency in one or more of (a) 
co gniti^^ e processing .^kflls. (b) perceptual enabling skills , or (c) 
neuromuscular vision skills . 

We believed that if appropriate, carefully designed software could be targeted 
at these underying levels that it might be \n appropriate way to demonstrate 
how computer tt^^mnology could be used * . i new and different way to aid 
learning disabled students. With this perspective, we could phrase the specific 
hypotheses and asscdated assumptions as 

1. The use of special purpose software that addresses cognihve 
processing skills can facilitate the reading of learning disabled children 

Assumption: The tmderlying perceptual enabling skills ard 
neuromuscular vision skills are in place. 

2. The use of special purpose software that addresses perceptual 
enabling skills can have an indirect, positive effect on the reading of 

^ learning .disabled children 

Assumption: The underlying neuromuscular vision skills are in place. 
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3. The use of spedai purpose software that addresses neuromuscular 
vision skills can have an indirect, positive effect reading of learning 
disabled children 

An important issue related to the development of these software prototypes 
involved the experimental determination of what features of instructional 
design ought to be embodied in effective software for the learning disabled. 
The partioolar features that were examined were: the use of on-saeen display 
of criterion performance, tiie ability for the learner to set the rate of 
presentation, the relative difficulty of exit criteria, and the provision of 
corrective feedback. 

Ftmding constraints and time limitations prevented a comprehensive test of 
the entire conceptual framework within a rigorous, concurrent experiniental 
design. Research on netiromuscular vision skills was conducted outside the 
CREATE study in a separate AIR project The initial experiment tiiat was 
imdertaken was at the enabling skills level, and the software research tool 
was called TurboScan . The second experiment was done with a software 
research tool called Wordsworth. 

Consideralle attention was given to practical design problems having to do 
witii lemming disabled students* ability to use tiie computer interface flexibly 
and in as transparent a manner as possible. Clearly, the evaluation of 
computerized instructional design must take into accoimt wether the 
students are forced to attend cognitively to the medium (hardware and 
software) more than to the message it purports to convey. Accordingly, these 
issues are described in some detail later in tiiis section. A brief description of 
each of the two software research tools follows. 



Description of TurboScan in its final form 

In its final research form, TurboScan consists of a program disk (side), a data 
disk (side) for scoring of students/classes, and a separate editing disk for 
creating new instructional screens. It is designed for use on the Commodore 
64 computer with color monitor. 

The main elements in the screen display are 

• a reserved upper area of the screen which shows 

- a selectable sprite (either robot, duck or none) 

- a set of empty squares symbolizing the nxmiber of targets 
to be found 

- a horizontal "tiiermometer" cumulatively displaying 
right responses (green segments) and wrong responses 
(red segments) and a criterion "star" to be reached at a 
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characters), choice ofsprites, and control of rate- Use of the editor is 
straightforward and requires no special computer knowledge. 

Description of Wordsworth in its final form 

The Wordsworth program consists of a set of disks containing program 
information on one side, with one "story" per disk, and individual student 
performance data on the other side. The program is designed for use with an 
Apple HE computer and color monitor. 

The main elements in the computer display are 

• a reserved upper area of the screen for presentation of directions 
and questions 

• a reserved lower area reserved for the presentation of text 

• a narro-* , horizontal band which separates the previous two areas 
and is used to present cimiulative scoring results 

• a "phantom" area which appears as an overwrite to the text for 
presenting memos and hints 

• two on-screen buttons for program control- "Hint" and "Done" 

• a bottom line reserved for reminders about additional "pages" that 
are available for view or review 

The instructional task is simply to read the text (as one would read pages and 
passages in in books) and respond to the directions/questions given. 
Questions are of two types: mark specific words or sentences that answer the 
question or choose the correct answer from among multiple choice 
alternatives. 

As will be described in greater detail later in this report, the studenc is to 
"solve" a mystery and earn "reward" points by answering the questions 
correctly. If an error is made, the individual gets a second chance, but at a 
lower level of reward. After two wrong responses the correct answer is 
provided. 

The computer record-keeping.system allows detailed tracking and printout of 
each student in terms of every key pressed and saeen display generated, and 
right/ wrong scoring as weU as a log of elapsed time. 



Resxdts of the 3rd Year Experiment 



Study 1: Assessment of Perceptual Enabling SkiUs and Improved Reading 

Study 1 was:conducted to evaluate the effectiveness of a first implementation 
of TurboScan in developing Reading-Related Visual Discrimination skills 
(RRVD) within a heterogeneous population of "learning disabled" children* 
tufboScan. the remedial reading software we designed for this project, reflects 
research^based principles drawn from two literattire reviews and is described 
elsewhere in this report. TurboScan is intended to develop low-^2vel visual 
discrimination skills characteristic of good readers and imderdevelpped in 
poor readers* 

Research design 

Study 1 was designed as a two-group treatment-control study, with subjects 
drawn from eight classes of learning disabled students at the Cupertino 
Union School District (elementary level) and five resource rooms at the 
Fremont Union High School District (secondary level), a total of 231 students 
(141 elementary and 90 high school). 

Participating students were pretested with a battery of instruments to establish 
a baseline for each learner. Included were tests of general ability, reading 
ability, reading-related visual discrimination, reading motivation, and 
motivation to use computers. Using the pretest and background skill 
information, the entire sample was sorted into strata using a nesting 
approach, with students first matched on teaicher, then on RRVD level, then 
general ability level, and finally by sex. Subjects were then randomly assigned 
to either treatment or control groups. Tests of group means showed no 
significant differences between the treatment and control groups. 

Students in the treatment group used the TurboScan software program 15 
minutes daily for six weeks. At the end of the treatment period, both 
treatment and control groups were given pdsttests to measure the following 
skills: eye movements (as measured by the Eye Trac apparatus), oral readmg 
speed (as measured by reading passages of the Gray Oral) and reading related 
visual discrimination skills (as measured by RRVD tests of the visual 
disaiminationskills taught by TurboScan) . The RRVD battery included: 

• Matching shapes. A match-to-sample exercise. 

• Finding reversals. Ciide reversed letters and numbers. 

• Matching short words L Match-to-sample two and three letter words. 
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• Matching letters. Match-to-sample small and-capital letters. 

• Matching short words JL Match-to-»umple two to fiv<» letter words. 

• Matching letters vrithin words. Match-to-sample within word. 

• Matching letter combination. Match-to-sample letter combination 
within a word. 

• Finding words of given length. Match-to-sample by word length. 

• Matching long words. Match-to-sample four to seven letter words. 

• Finding the word that is different: Identify non-matching word. 

• Make words using letter combination. Spell word by inserting letter 

coinbination in blanks. 

Results 

An analysis of covariance was used to estimate the size and significance of the 
treatment effect and was conducted in two steps. (1) A set of background 
variables to be used as covariates was identified, and (2) an analysis of 
covariance was conducted on each set of criterion measures of interest. 

For the elementary school data. Table 1 shows the correlations between the 
candidate backgrotmd measures and the posttest RRVD skills measures. It 
was dear that age and reading ability are correlated with all deven RRVD 
teste; none of the other backgrotmd variables are related. 

T-»e two background variables, age and reading ability, wei'e included in the 
analysis of covariance. A multivariate test evaluating the entire battery 
jointly was, significant at the .05 level. Table 2 summarizes the results for the 
individual variables. 

The same analytic procedures were carried out for the secondary school 
sample. A multivariate test evaluating the entire battery jointly was not 
significant Table 3 summarizes the resulte for the individual variables. 



Summary 

The primary objective of Sti^dy 1 was to determine whether the use of 
TurboScan produces measurable gains in visual disaimination and reading 
skills. This objective was met to a limited degree. Although the analyses of 
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Table 1. Correlations of backgrour d variables and RRVD tests: 
deanientary school sample. 

SRVDTest Age Slossen Woodcock Motivation Motivation 

to read for computer 

Shapes 
Seversals 
3hort words I 
Letter! 
Short words II 
t^tters.n 

Letter combinations 
"Word length 
Long words 
. Identify different itenu 
Make words 



26 


.04 


.40 


-.07 


-.13 


.26 


-.05 


.25 


.03 


-.04 


.23 


-.io 


.39 


-.01 


-.05 


31 


.01 


.39 


.10 


-.03 


29 


-.05 


.54 


.04 


-.02 


38 


.01 


.38 


.01 


-.09 


34 


-.03 


.40 


-.06 


-.08 


.39 ^ 


.14. 


.49 


.07 


: .00. 


.28 


-.12 


.44 


.03 


-.02 


.32 


-.01 


.46 


-.04 


-.03 


.35 


.06 


.39 


.02 


-.05 
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Table 3. Analysis of lieatment and control differences: Secondary data 



D«p«nci9nt 



Troatment Comparieon 



Tdtts of Adiusted 
Itoan DlllorGnces 



Analysis oi 

Within*group nelatlons 





U'ian 


S.D. 


Mean 


S.D. 


F 


p Isvel 


H 


r 


p Hvot 


Matching Shapes 


o.2o 


no 


ft 91 


1 06 


.01 




.18 


3.60 


.01 


Finding Ravsisais 


11.29 


2.85 


44 4 4 
11.11 








.16 


3.22 


.02 


6Aalchino Shoit 


5.81 


.56 


5.80 


.58 


.02 




.12 


2.28 


.07 


Wocdii 






















9.10 


.92 


9.17 


1.10 


.48 


— 


.09 


1.72 


.16 




6.81 


.46 


6.63 


.60 


3.11 


.08 


.21 


4.35 


.01 


UakMid LttttQfsin 
NontanM SySaiites 


22.71 


3.38 


22.03 


4.36 


.18 




.17 


3.43 


.02 




13.53 


2.00 


13.23 


1.68 


.35 




.13 


2.52 


.05 


FincSnoytordBOl 


13.37 


3.15 


13.09 


3.31 


.01 




.17 


3.47 


.02 






















iuUkiibig Medun>-Ungth 


6.34 


.88 


6.29 


1.02 


.01 




.14 


2.70 


.04 






















Fk)dir«Wudlhali8 


8.50 


1.35 


8.31 


2.03 


.04 




.17 


3.39 


.02 


OMforanl 




















Ma((9 Words with 


10.26 


3.65 


9.77 


3.07 


.17 




.34 


8.78 


.01 


UttMPaks 




















1 

Gfsy Oral Total 


77.53 


33.75 


75.29 


21.73 


.01 




.S» 


17.04 


.01 



Note- Eacfa r2 is the proponioo of v«i»ce ia the (Sepexieu variable ihai is expJainrd by ibe Uaear itgreaioo of the dcfcM variabJc oo age, WRAT, Slosscn. a«d 
Otfei. Tl»5»e ««ly»e» w teed 00 M C3S» io 6» »e«iacM group ai^ 
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the data did p.-oduce a significant effect in favor of the tra^tme-At, the size of 
the effect waa sv^naller than expected, primarily due to problems in 
instrumentati,.rv we believe. Because the RRVD pretests shv)wed a 
pronounced c^iiing effect, it was necessary to use shorter time iiiruts on the 
posttesc. This led to a posttest-' -nly design and a weaker analytic technique. 
We were encouraged that a significant effect was found fcr the elementary 
school samui«i. 

We also learned that the measurement of eye movements in this population 
is more problematic than originally thought. Consistency among the raters of 
the Eye Trac printout was not as high as we expected and we therefore 
standardized the Eye Trac scoring procedures so that similar problems would 
not occur in Study 2. We also gained some ideas about how to improve 
TurboScan, particularly in making the task more motivating and challenging 
to the student. 



Study 2: Evaluation of the Irstructional Impact of TurboScan Features 

Study 2 was a further investigation of the effectiveness of TurboScan in 
remediating reading skill deficiencies. A new'research objective was added, 
tiie research design was modified, and significant changes in instiimientation 
were made. Five treatments similar to those of Stiidy 1 we: ? evaluated, usmg 
tiie conb-ol students firom Study 1 as tiie treatinent group. No control groups 
were used. Refinements in the RRVD skills batteries and standardization of 
procedures in tiie administration of Eye Trac and Gray Oral instruments 
resulted in more consistent and sensitive measures of RRVD skills, eye 
movements during reading, and reading speed. 

Research design 

Subjects were heterogeneous with respect to reading ability and scores on 
inteUigence tests. At the elementary level, groups of subjects witii the same 
teacher were assigned to specific treatinents. Use of the TurlpQScan software 
program for 10 tol5 minutes each day for six weeks with five different 
program variants. Assignments and conditions are shown in Table 4. 

At ti\e secondary level, the subjects (N=33) received the conditions Z4 and 2.5 
described above (N=13 for 2.4 ; N=20 for ?-5). 

The five different treatinent conditions were aeated by modifymg the 
software in selective ways to address questions about generalizable program 
features titat were considered to have a potentially significant impact on the 
effectiveness of the hreahnent. 
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Table 4. Assignment of subjerfe 'co conditions 



(Elementary level) 




Teacher 


N 


Study no. 


Conditions 


2.1 


No bar graph showing accuracy 


1,2 


IS 


2.2 


No-feedback on errors of omissjon 


3,4 


24 


23 


Learners can set their own speed 


5 


17 


2:4 


Relatively easy exit criterion 


6 


12 


23 


Relutively hard exit crterion 


7 


11 


(Secondary level) 








2.4 


Relatively easy exit criterion 


8 


13 


23 


Relatively hard exit criterion 


9 


20 



(1) Does the use of a visual indicator giving continuous information on 
accuracy of. learner responses and on overall -performance serve to shape subject 
response patterns in a beneficial way? A bar graph or "thermometer" was 
included in the upper part of the saeen to provide immediate visual 
evidence of performance relative to the criterion. It provided 
aamulative information on the proportion of right and wrong 
answers. 

(2) Does knowledge of specific omissions, coupled with additional opportunities 
to find those targets, increase the effectivenesa of the software? A feature was 
added to highlight missed targets in red following the completion of 
scoring for each saeen. 

(3) Does learner control over the ra te of presentation improve .he effectiveness 
of the treatment?- Subjects were able to-set and change the speed at 
which the scanning window wcvald move, rather than having the 
automatic adjustment used iti Study 1 and for other conditions in 
Study 2. 

(4) and (5) Does the difficulty of the exit criterion affect the performance of 
subjects and the rate at which they are able to learn RRVD skills? The exit 
criterion was made easier in one version and more diffiaUt in the 
other. 

Gains were evaluated using a pretest-posttest design, with Study 1 posttest 
measures for the control group used as the pretest measure for this study, and 
three sets of deper^dent measures for the post-test: eye movements (as 
measured by the iEye Trac apparatus), reading-related visual discrimination 
skills (as measured by RRVD tests of the visual discrimination skills taught by 
TurboScan), and oral reading speed (as measured by reading passages of the 
Gray Oral). 

Results 

For the elementary school students. Table 5 summarizes the results for the 
Gray Oral and the RRVD measures. All measures are statistically significant 
at the p<.01 level or greater. On the average, elementary school students 
gained on both the visual discrimination and reading tests. 

Because the Gray Oral and RRVD tests are on different scales of 
measureinent, it is not easy to judge the magnitude of the effects from 
inspection of the means. To make this easier, we divided each average gain 
score by its standard deviation. This statistic is reported in the last column of 
Table 5 and graphed in Figure 2. 
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Table 5. Pre- and posttest means on Gray Oral and BKVD .ests: 
elementary data 



Variable 



GrayC)ral 
Paragraph 1 
Paragraph 2 
Paragraph 3 




76 

75 
72 

72 



76 
76 



.97 
1.29 
1.44 



.86 


7.68" 


.83 


1.24 


4.58" 


.53 


1.34 


7.49" 


.89 



Total 
RRVD Tests 

Matching Shapes 
Finding Reversals 

Matching Short Words I 77 

Matching Letters 77 

Matching Short Words II 77 

Matching Letters 77 

Matching Letter Pairs 77 

Rnding by Length 77 

Matching Long Words 77 
Rnding Different Words 77 

Making Words 76 



No.e:Tb.mean,lor,h«G«yO«lte85we«»mpu.e<ifromm.lo,anftm.l.h.tlme 
to road each paragraph. 



1.75 


1.66 


8.50" 


.99 


5.34 


6.38 


8.52" 


.98 


8.39 


9.60 


4.68" 


.54 


4.92 


5.40 


3.62" 


.41 


7.14 


8.05 


7.48" 


.85 


6.02 


6.29 


2.73" 


.31 


18.11 


21.01 


7.94" 


.90 


9.60 


11.62 


8.41" 


.96 


9.87 


12.43 


8-00" 


.91 


14 


5.54 


2.83" 


.32 


6.06 


7.36 


6.47" 


.74 


8.13 


7.67 


4.95" 


.57 



•♦p<.oi 
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36 



1.00 - 



If- 



Gain in 

Standard 

DeviatioQS 



to 
o 
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.75 - 



.50 - 



.25 - 




37 



JL 



Gray 



Key Gfay Oral Passages: 

IrPrc-Primcf 
2: Primer 
3:FiislGrade 
T: Toial of the above 



RRVD Skills: 

1: Matching Shapes 

2: Finding Reversals 

3: Matching Shoit Words 1 

4: Matching Leuers 

5: Matching Short Words H 



RRVD Tests 



6: Matching Imbedded Letters 
7: Matching Imbedded Letter Pairs 
8: Finding Words of Given SLength 
9: Matching Medium-Length Words 
10: Finding Non*Matching Words 
1 1: Making Words Using Utter Pairs 



Figure 2. Gains on the Gray Oral and RRVD tests: Elementary data 



10 



II 



IS 



Gray 



JL 



10 



11 



RRVD Tests 



Key Gray Oral Passages: 

1: Prc-Primer 
2: Primer 
3: Firsl Grade 
T: Total of the above 



RRVD Skills: 

1: Matching Shapes 

2: Finding Reversals 

3: Matching Short Words I 

4: Matching Letters 

5: Matching Short Words !I 



6: Matching Imbedded Lettos 
7: Matching Imbedded Letter Pairs 
8: Finding Words of Given Length 
9: Matching Medium-Length Words 
10: Finding Noo-Matching Words 
1 1: Making Words Using Letter Pairs 
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Figure 2. Gains on the Gray Oral and RRVD tests: Elementary data 



4 0 



Table 6 summarizes the results for the Eye Trac Data on the subset of the 
elementary school sample for which we were able to obtain valid pretest- 
posttest data (some students could not be calibrated on the Eye Trac devjce). 

The results indicate that TurboScan had an impact on eye movement 
patterns. However, we have no data for the group of students whose eye 
movements are known to be tlie most eixatic, so extrapolation of these results 
to popxiiations with different characteristics is not encouraged. 

The effects of variations in the features of TurboScan were measured by 
comparing differences among gain scores. Table 7 reports on tl>e significance 
of differences among gain scores, with t-test entries in the first five colunms 
to indicate whether the gain associated with particular condition is 
significantiy different from the average of the gsdns of the remaining 
conditions. 

Table 7 does not show any strong trends. None of the RRVD tests indicate 
differential gain. One of the three Gray Oral tests, the first-grade passage-does 
show differences in gain that "approach" significance. We conclude that no 
strong evidence favors one particular variation over another. 

For secondary school students. Table 8 summarizes the Gray Oral and RRVD 
results. Gains for the secondary sdiool students were statistically significant 
for all three Gray Oral passages and for nine of the eleven RRVD tests. The 
two RRVD tests not showing significant gains both require students to match 
target words. Because the remaining such RRVD iesl does show significance, 
it is not clear why these two tests do not show significant gains. 

Valid gain scores on the Eye Trac were obtained for only twelve secondary 
school students. No improvements in fixations or regressions were found. 
These levels correspond fixactly to the posttest status of the elementary school 
students that were assessed using the Eye Trac and may represent some sort of 
ceiling. 

Comparisons of the means on the RRVD and the Gray Oral for the "Easy Exit" 
and "Hard Exit" conditions did not reveal any consistent patte: s, so we do 
not draw any conclusions from the results. 

Summary 

Study 2 established that TurboScan is an effective remediation tool. 
Significant gains were found on all three sets of outcome measures. Evidence 
- - was gained that the effect of d'^iiig TurfefiSsaii exercises on the computer does 

transfer to both visual discrimination paper-and-pendl tests and oral reading 
speed. Differential effects of TurboScan by varying certain features of the 
software were not demonstrated. 
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Table 6. Pre- and posttest means on Eye Trac Measures: Elementary data 



Number 
of 

Variabid Cases 




Means ■ 




Effect 


Pretest 


Posttest 


t-test 


Size 


Fixations/100 words 32 
Regressions/lOO words 32 
Per'Cant Regressions 32 


170.00 
46.90 
.27 


150.63 
' 35.06 
.23 


3.42" 
2.78" 
2.08* 


.60 

.48 ^ 
.37 



Note: The tests of significance wars carriad out on tha togarithma of th« coums. 



"p<.Ol 
•p<.05 



3^2 



Table 7. Comparisons among TurboScan treatment groups 



□•Pendent 
Vsriabi^ 


NO 

Therm 


Easy 

Exit 


nirvl 

Exit 


R«d 


w n ii i 1 y w 

speed 


P-Statlstic 




-.48 


.51 


l.od 


1 Q9 


.4/ 


2.79 


(.06) 


Gfay6ral2 


.39 


1.48 


•1 .46 




• 1. lO 


1.27 


(.29) 




.76 


3.28 


-1.42 


•.30 




3.78 


(.01) 


Matchiny Shapes 


1.29 


•1.11 


.27 


4 AA 

1.UU 


-1.10 


1.09 


/ 'O \ 

(.36) 


Finding Reversals 


.S3 


.32 


i id 




-^i>.30 


.99 


(.42) 


Short Words 1 


-.89 


2.39 


- 1, i4 




.10 


1.59 


(.19) 


Matching Letters 1 


-.39 


1.63 


4 A4 

-1.01 


.09 


•1. lo 


1.17 


(.33) 


Short Wonis II 


.13 


-1.13 


-.89 


.36 


1.S3 


1.17 


(.33) 


Matching Letters II 


1.18 


1.75 


-.93 


•1.77 


-.45 


1.76 


(.15) 


LetteTr Pairs 


.£0 


-.62 


.36 


-1.05 


.77 


.52 


(.72) 


Word Length 


-.76 


-1.13 


1.33 


.17 


.28 


.72 


(.58) 


Matchk^ Words III 


.37 


2.00 


-.96 


-.39 


-1.19 


1.28 


(.28) 


Different Word 


1.56 


-.28 


-.57 


-.94 


.37 


.?2 


(.52) 


Make Words 


-.60 


.39 


1.59 


-rss 


-.38 


1.0S 


(.39) 



Notes: Eiuries are t*teso that tat esch subgroup aaainst tbe average of the remaining groups. A t*test that is 
greater than 100 is significant at tbe .05 level 



Table 8. Pre- and posttest means on Gray Oral and RRVD test: Secondary data 



Variable 



Number 

of 
Cases 



Means 



Gray Oral 



Pretest 



Posttest 



t-test 



Effect 
Size 



Paragraph 3 


32 


1.27 


1.23 


4.12" 


.73 


Paragraph 4 


32 


1.40 


1.37 


3.46*^ 


.61 


Paragraph 5 


32 


1.48 


i.a 


3.3r* 


.SO 


Total 


32 


1.87 


1.83 


4.56" 


.80 


JRVD Testa 












Matching Shapes 


33 


6.18 


6.87 


3.83" 


..67 


Finding Reversals 


33 


.11.18 


12.36 


2.74" 


.48 


Matching Short Words 


1 33 


5.70 


5.94 


2.27* 


.59 


Matching Letters 


33 


9.24 


9.36 


.68 


.12 


Matching Short Words 


ii 33 


6.51 


6.94 


3.9r* 


.69 


Matching Letters 


33 


22.36 


24.03 


3.44" 


.50 


Matching Letter Pairs 


33 


13.42 


14.42 


2.81" 


.49 


Finding by Length 


33 


13.24 


15.09 


4.33" 


.75 


Matching Long-Words 


33 


6.36 


6.45 


.53 


.09 


Finding Different Wordr, 33 


8.24 


8.94 


2.7T' 


.48 


Making Words 


33 


9.97 


12.58 


5.51" 


.96 


Nolo: Tha mar» fw th« Gray Oral tests war* computed frons tha togvtthm of the time 
to read each parat ifti. 



•p<.(j5 
"p<.01 



34 
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Study 3: Using TurboScan to Remediate Severe Learning Disabilities 



Study 3 was designed to investigate the efficacy of TurboScan in the 
remediation of fairly severe learning disabilities* The major goals were to 
understand how members of this population interact with particular kinds of 
material that can be taught by TurboScan. 

Research design 

A case study methodology was used because the kind of information sought 
required a detailed analysis of the relationship between student backgroimd 
characteristics and performance and because very few students were available 
to AIR who met the criterion for inclusion s the study. 

Study 3 differed from Studies 1 and 2 in four ways* 

(1) The target population consisted of the weakest students who are 
generally placed in remedial reading programs. 

(2) The most complete version of the TurboScan program was used 
(including features added in Study 2). 

(3) Entirely new saeens were written, limiting the textual content to a 
pre-selected vocabulary and presenting it in a newly-defined 
instructional sequence. 

(4) Students met for one hour a day, ♦Ive days a week, for one month. 
The session took place during the summer break; students were not 
attending regular school at the time. 

Outcome data gathered in Studies 1 and 2 were used to identify students with 
serious reading difficulties. Students who could not read a first grade level 
paragraph.sample on the Eye Trac device with at least 50% comprehension 
were nominated; nineteen students met the criterion; eight were 
volunteered. Three other students, siblings who were weak readers but 
stronger than those who met the criterion, were added subsequently. 

The sessions were conducted by a resource specialist from the cooperating 
school district; an AIR staff member served as an aide. Testing took 
approximately two days at the begirming and end of the session, and each 
student received between 8 and 12 hours of actual exposure to TurboScan . 
The skill categories covered in the treatment and the niunber of screens 
contained in the curriculum are shown in Table 9. 



Table 9- TuiboScan^skill categories used in Study 3 



Category Number of saeens 

Inidal letter 20 

Inidal letter combination 10 

Word length 10 

Word length and initial letter 10 

Word length and intial letter combination 10 

Shapes (2 and 3 letter words) 10 

Shapes (4 letter words) 10 

Shapes (5 letter words) 6 
Shapes (2 t^get shapes per .saeen, with word length constant) 12 

Shapes (2 target shapes per iscreen) 30 

Shapes and initial letter 10 

Comprehension (Find word according to category) 30 

Find and say target word 100 
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The following outcome measures were used. 



• Six RRVD Tests were administered: match short words, match long 
words, find words that start with a 2-letter combination, find words of 
given length, find the word that is different, and make words with 
letter combinations. 

• The pre-primer, primer, first- and second-grade passages from Form 
B of the Gray Oral test were administered. Students were tested 
individually on the RRVD tests and the Gray Oral. 

• The Eye Trac measure was attempted, but we soon realized that the 
test was unsuitable for this group due to our inability to get reliable 
data on eye movement 

• Variables that described studevAts' performance on each saeen were 
automatically recorded by the TurboScan program. On the basis 
primarily of the pre-primer passage on the Gray Oral, students were 
grouped into three subgroups: readers, intermediate readers and non- 
readers. 

Results 

Given the individual differences in reading that were evident in the three 
subgroups, it is not surprising that the students were able to complete a 
different number of screens in the .TurboScan program during the course of 
the summer class. The order of saeens was predetermined because of 
assumptions about task difficulty, so it is of some interest to report, in Table 
10, how far each stud^t got. 

Tables 11 and 12 below contain the results of the group level analyses for the 
Gray Oral and RRVD tests. Although the number of observations is small, 
gains on the first three Gray Oral passages and three of the sbc RRVD tests 
were-statistically significant. Despite the wide range of ability levels within 
the group, results showed dear and convincing evidence of gain. 

Data on several process measures were provided by the Turb<??can program. 
The program Will maintain records of students' progress and describe how 
well individual students do on particular kinds of exercises. Four variables 
were recorded: (1) total time spent on each saeen (derived by adding aaoss 
any multiple passes or saeen "tries," (2) number of ities needed to re?ch the 
criterion, (3) number of targets and number of them found on the try that 
met the exit criterion and (4) number of incorrect targets marked on the try 
that met the exit criterion. Because of problems with missing data during 
computer recording, results are approximations and may be somewhat 
affected by attendance problems. We combined two original variables. 



Table- 10. Niiiaiber of saeens completed by Study 3 students 



Student 

Readers 
Lori 
Mark 
Jenny 



Last category studied 



Find and say 
Find and say 
Find and say 



Intermediate readers 
Karyn 
.Marie 
Jason 
Ellen 
Bob 
David 

Non-readers 
Bill 
Start 
Joshua 



Find and say 
Find and say 
Find and say 
Comprehension 
Shapes (2 per screen) 
Shapes and initid letter 



Comprehension 
Find and say 
Shapes (2 per screen) 



No. of saeens studied 



230 
220 
210 



220 
230 
180 
170 
95 
138 



146 
155 
103 



38 



48 



Passage 



Table 11. Study 3: t tests on Gray Oral passages 



Pret. 



Posttest 



t-test 



Level 1 
Pre-prlmer 



Median Tune (Seconds) 26.00 

Mean Log of Tune 1.48 

S.D. .44 
N-12 



17.67 
1.33 
.48 



5.07" 



Level 2 
Primer 



Median Tune (Seconds) 48.00 

Mean Log of Tune 1.71 

S.D. .39 
N-IO 



32.10 
1.55 
.35 



4.70« 



Level 3 
First Grade 



Median Tune (CecoDds) 81.00 

Mean Log of Tune 1.83 

S.D. J5 
N-9 



53.00 

1.65 
.32 



527*' 



JLevel 4 
Second Grade 



Median Tune (Seconds) 77.00 

Mean Log of Tuns 1.82 

S.D. .35 
N-8 



62M 
1.75 
.28 



1.97 



•p < .(»5 
••p.<.00l 



Table 12. Study 3: t te5ts on the RRVD measures 



JR.?IVD Test 




Pretest 


pQsttest 


t-test 


QhAPt Wa»/4« 


Median 


11.5 

IZ5 
4.7 


13.7 
12.5 
6.2 


2.40- 


Maectt Long Words 


Mean 

Median 

S.D. 


9.7 
8.5 

5.6 


11.3 
6.2 


2.66- 


Match bj Initial 
Letter P^ir 


Mean 

Median 

S.D. 


10.1 
9.5 
3.6 


11,2 

lOJ 
4.6 


2.76* 


Fiod the Word that 
is DiffereDt 


Mean 
Medial 

s-a 


8.7 
8.0 
3.0 


7.8 
8.0 
3.0 


-1.20 


Find Words of 
Given Length 


Mean 
Median 


20.6 
20.5 
4.3 


20.7 
20.5 
4.6 


.12 



Make Words Using Mean 3.4 4.2 1 29 

a Letter Pair Median 2.0 3^ 

S.D. 2.6 3^4 



NouK N - 12 
• p < .05 



number of targets and the number of targets found, into a single measure, 
percentage correct, and we computed means for each variable, by skill 
category, as shown in Table 13 below. 

Overall, the pattern of results is consistent with intuition. Within nearly 
every skill category, the expected rank ordering of the subgroups is consistent 
with the actual ordering. The performance of all thres subgroups improved 
as they proceeded through the three kinds of "length** screens; the ten screens 
that teach the ^'shapes of two- and three-letter words'* proved to be the most 
difficult saeens in the entire curriculum. 

The three readers and two in the intermediate subgroup did well on the 
comprehension screens; non-readers and poor readers developed a 
compensatory strategy that enabled them to do the saeens without knowing 
how to read any of the words. 

For "Find and Say," the better readers performance clearly improved. For the 
worst students, those saeens were ineffective. For the top five students, the 
goal of completion of the "Find and Say" saeens-was reached. 

Summary 

Our outcome data show that practice i n TurboScan transfers to similar paper- 
and-ji^adl tests (RRVD tests) and improves oral reading speed. Although the 
sample was small, pre- to posttest gains were highly significant We believe 
that the general efficacy of TurboScan cannot be seriously challenged. The 
Gray Oral data indicate that, excluding the non-readers, gains in reading speed 
are fairly uniform, while the RRVD data suggest that gains on skills taught by 
TurboScan are positively related to initial abiUty. Our data also suggest that 
the Gray Oral can be a useful saeening device. 

The low ability, non-readers pose two unique instructional problems: (1) 
Non-readers need intensive instruction in decoding skills and we cannot 
identify an existing computer program that can provide this instruction. 
Although non-readers might benefit in some ways from work on TurboScan. 
acquiring basic decoding skUls is a higher priority for their classroom study. 
(2) Non-readers in our sample have been able to build up maladaptive 
compensatory strategies to cope with schoolwork that they cannot do. These 
strategies are a kind of outer "shell" that any effective instructional program 
of remediation must penetrate. In its present form TurboScan cannot 
provide this Jype of remediation. 

Both non-readers and the intermediate subgroups had considerable difficulty 
with the advanced word length and :vord shape saeens, a ^;oblem that seenis 
to be associated with left-handedness and large WISC Verbal-Performance 
differences. This suggests the need for a specialized curriculum that 
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Table 13. Means by skill category for TurboScan process variables 



Frocotft 
Variable 



TurboScan^ S&IU C^Ugory 



Group 

# 

TImt par ^ 


lei. 


fnitkJ 

let. 

Cccnb. 


Length 


length/ 
inilLot 


len0h/ 
tnillet 


Shapes 
(243) 


Shapes 
(4) 


Shapes 
(5) 


Sh«4>e 
Cotntin. 


Shape/ 
I(i5tlet. 


Comixe- 
h:K)sion 


Find& 
Say 












41.33 


ce C0 


56.25 


44.78 


48 38 


93.75 


33.25 


46.41 














CC £A 

ob.oS 


26.77 


34.67 


32.47 


34.40 


32.95 


26.19 


RfiAdora 


^.37 




27.61 


23.10 


19.60 


35.67 


33.24 


2725 


^.46 


32.04 


2652 


16.48 


(ifotmboc* of : 
AttdiapU 




















































2.03 


2.33 


4.17 




3.44 


3.89 


4.92 


1.61 


2.61 


4.94 


5.62 


3.18 




1.4d 


1.47 


3.37 


1.45 


2.06 


4.47 


3.30 


1.67 


2.10 


2.60 


2.37 


1.70 




1.78 


1.75 


2.18 


1.77 


1.90 


2.67 


2.75 


1.75 


1.60 


2.18 


1.98 


1.87 


P«r Cent 
Correct 


























M 


.04 


.94 




.94 


.88 


.92 


.97 


.95 


.91 


.94 


.99 


IntsrmMiiate 


M 


.97 


.96 


.96 


.96 


.94 


.96 


.94 


.94 


.S3 


.93 


.90 


Readm 


.65 


.97 


.96 


.94 


.94 


.85 


.91 


.97 


.95 


.94 


.92 


.97 


dumber o& 
Wrong 


























Keyproeaes 




























1.07 


2.87 


2.03 




1.78 


1.78 


154 


.72 


1.13 


1.72 


275 


.29 


faUcfmediale 


.91 


1.74 


1.96 


1^ 


1.43 


1.28 


1.60 


U1 


1.31 


1.47 


1.72 


.41 




1.45 


1.40 


1.76 


1.53 


1.60 


2.00 


1.43 


.75 


1.12 


178 


1.28 


.43 



emphasizes length and shape skills to a greater degree than was used in the 
experimental curriculum. 

TurboScan offers an automated way of teaching visual discrimination skills 
to children. Based on this research, it now seems possible to predict gain 
reliably in terms off visual disaimination ability. There is evidence to 
suggest that a number of students are better able to read or?lly following 
TurboScan study; It-is also-possible to describe subgroups of learning disabled 
children, who need intensive work on particular skills that TurboScan 
teaches. Finally, it is also possible to identify a subgroup of nonreaders for 
wrhonj a more intensive program of remediation is indicated than can be 
provided by TurboSc. n . 
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Optimizing the Student/Machine Interface 

The general instrur.aonal strategy was to cause students to attend carefully to 
facts and details iii the presentation of several detective "cases." Three 
detective stories were used as a primary means for developing students' 
attention to facts and details within narrative text. 3y graduating the 
diffictilty of the questions within.and across the detective cases it was expected 
that students would improve their ability to recognize facts and details that 
are frequently imbedded in other information. 

The students' attention to detail was required to prepare for questions 
interspersed through the tory. By presen'lng the information as dues In the 
context of a problem to t< "solved," this attention to detail seemed jtistifiable 
to the students (very protjably more so than would be expected to occur 
during the silent reading of standard passages in school texts). 

In each case the student piayed the part of a junior detective and "solved" the 
case. Each student was presented blocks or "pages" of information about the 
case and then asked to identify specific facts or details. Depending on the 
question, identification was accomp ished either by selecting the beic answer 
from among four multiple-choice options or by marking the sentence within 
the text that actually presented the fact referred to in the question. 

During the presentation of the cases, each inaement of information (termed 
a lesson) was scored by the computer whenever the student indicated he or 
she was satisfied by the answer that had been selected and highlighted. 
Scoring was activated by pressing the spacebar. Students could change their 
minds and freely select different answers prior to preying the spacebar. Five 
"dollars" were awarded for correct answers on the first try, three "dollars" 
were awarded for correct answers on the second try, and no "dollars" were 
awarded if the question was answered wrong twice (the correct answer was 
automatically displayed in this situation). 



Critical Design Issues Involving tlie Student/Machine Interface 

In the early stages of planning for the CREATE research, project staff felt that 
it was important to make the interface between the student and experimental 
programs as "transparent" as possible. In the prior year's research with 
Turboscan software, designed to develop students' perceptual enabling skills, 
the "press any key" interface greatly simplified students' responses, which 
had to be quick and accurate. This strategy could not be tried in Wordsworth. 

In the Wordsworth software, which was designed to develop cognitive 
procej-3ing skills, the goal was to make the software as ^ansparent as possible 
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but also to give the student a great amount of flexibility nnd control over the 
content being presented. To accomplish this, we initially planned for 



• Separate display "windows'* for 

(a) textual information/ 

(b) directions, queistions and reinforce- -jit statements, 

(c) "feedback" ccirective memos. and ^'nts 

• Scroll capability on each of the windows so that additional text 
could be accessed forward and backward in a maimer similar to 
some word processors. Access was accomplished by registering 
the ciirsor oyer upward and downward pointing blinking 
arrows (buttons) located in.the margin of each window and 
then depressing a control button on the joystick. 

• On-saeen "buttons" for management of program functions. 
Six were envisioned, including 

(a) Ifint- td request a- hint 

(b) Done- to indicate that the day's study is finished 

(c) Next- to activate scoring and iidvance to the next lesson 
id) Undo- to "erase" a mistaken answer 

(e) Mark- to identify targe*^ words or sentences m text 
(£) Move- to extract a target wdfd or sentence from one 
location and insert it at another location 

« Cursor control via a jo3rstick, allovnng direct movement 
(horizontal, vertical, diagonal) to particular locations on the 
saeen. Al-'ematively, the cursor could be controlled from the 
keyboard. The cursor itself was in the shape of a pointing hand. 

« Student-controlled highlighting of text to prepare for the 
traiisfer of words, phrases and sentences from one location to 
another or for the cutting and inserting of text (the cat and 
insert capabilities were never implemented in the detective 
stories dealing with facts and details but were incorporated in 
the program so they could be used with subsequent research 
studies dealing witii sequencing and maui ideas.) 



PreUminary Pilot Trials of the Interface 

As a check on the usability of the saeen design and ttie joystick input device, 
the program was tried out with younger students (5th and 6th grade) and with 
a few adults. It was immediately apparent ti\at the initial configuration 
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(described .above) was J^it ei workable interface. The following featxires were 
targeted and changed based on < observation of . students* problems: 

• Cursor control; The joystick proved to be an inefficient and 
unreliable method for controlling the cursor tinder conditions 
where specific lin^ of text had to be parked or specific buttons 
activated on^the screen. (Three models of joysticks were tried - 
all were so erratic and uncontrollable that they frustrated the 
students and distracted from the lesson.) 

Accordingly, the joystick was replaced with a mouse. The use of 
keyboard entry was sharply reduced; only the spacebar, return, B 
(back), N (next) keys continued to be used 

• Screen. Four of the sbc control buttons were eliminated, 
retaining only HINT and DONE. The windows (for directions 
and questions, for text, and for corrective feedbadc) were 
Increased in size, allowing more information to be seen at a 
time. 

In the revised version, special instructions were caused to appear 
at the middle of the saeen or at the bottom of the saeen under 
specific conditions. For example, if there were additional pages 
of textr a bottom line would. appear informing the student of the 
Next or gack page-turning options. 

• Text marking. The use of diagonal marking action was 
originally expected to allow direct moveme:at from tlie 
beginning to the end (or vice versa) of phrases or sentences in 
the same mann^ as the with tlie Macintosh. This non-linear 
movement confused the students. 

Next, a simpler means of marking was tried in which the 
highlighted text would automatically wrap to the ne^*: line. This 
was conceptually more straightforward, but still required a 
degree of motor skill that got in the way of the cognitive process 
of readmg the text and comprehending it. In adddition, students 
were experiencing difficulty in isolating specific words or phrases 
because program ming required that only the relevant text 
marked in an answer. 

For these reasons, the text marking procedure was further 
^ - ~ liberalized to automatically highlight entire sentences when any 

part of the sentence had been pointed to and marked by 
depressing the mouse button. 
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• Text Handling , The scrolling feature proved to be less than 
satisfactory in the trial with younger student??. It was difficult for 
students to orient the cursor oh the up/down arrow and hold it 
steady dtiring-the time the scrolling was taking place. The 
optimal rate of saolling varied considerably across students, so 
that the actual setting of the rate represented a delay for some 
students but was too fest for others- In addition, the students 
would sometinies quit scrolling before the end of the block of 
text in which the correct answer was imbedded, with the 
consequence that they would tsy to select an answer based on 
incomplete information. 

Because of this, the procedure for viewing additional text was 
changed from scrolling, to a student-controlled page-tuming 
action, activated by pressing the N(ext) or B(adc) keys. This 
presented new information in increments of up to eight lines of 
text A check was built into the program to ensure that all pages 
had been read before the question could be answered. 

It was apparent that this experience with interface design and redesign was an 
important and unanticipated event with considerable import for any 
developers of software aimed at the learning disabled. 
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Resxilts of the 4th Year Experiment 



Research Design: Cognitive Processing and Reading Comprehension 

The experimental design called for a comparison of matched randomly 
assigned treatment and control groups. Pre and posttests were administered 
to both groups. The control group then began computer study after the 
posttest was completed. They then took a third test. This design allowed 
examination of treatment effects with matched groups (subsequently referred 
to as the First Analysis) and also with each student serving as their own 
control (subsequently referred to as the Second Analysis). 

All students were administered the Comprehension subtest of the Stanford 
Achievement Test, Intermediate level 1, Form E, (1982 edition) prior to the 
initiatio.1 of the training. Students were then matched according to their 
score- on the literal subscore of the comprehension subtest and randomly 
assigned to the treatment or control condition. Posttests consisted of Form F 
on die same test. 



Student Characteristics 



Twenty six students began the fourth year Wordsworth study. One student 
who was assigned to the control group dropp)ed out due to illness and other 
family problems. Table 14 shows the grade assignments of the participants 
and the relative severity cf their learning disability. 



Table 14. Grade Levels of Participants and Severity of Learning Disability 

Grade Severity 

Study #1 

Treatment (N=13) 
Control (N=12) 



9 


10 


11 


12 


moderate 


mild 


4 


3 


1 


5 


5 


8 


3 


2 


4 


3 


1 


11 



Participating students had been previously identified by the school disbiict as 
learning disabled (not necessarily reading disabled) and were either in a 
resource program (niildly handicapped) or in a special day class (moderately 
handicapped). Six of the 25 were moderately handicapped. It should be noted 
that the procedure for assigning students to the treatment and control groups 
was random (after matching according to the score for the literal factor in the 
reading comprehension pretest). However, using the random proc^ure, 
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five of the six moderately handicapped students were assigned to the 
treatment groupt 

Treatment Environment 

Three Apple II E computers were placed in two classrooms and one 
counseling center, at the Cupertino High School, Fremont Union High 
School District, California. Each computer was equipped with a color 
monitor, disk drive and mouse. 

Students were scheduled to use the computers on a rotation basis over the 
course of a six-period day. Supervision of the computer training wai by two 
or three AIR staff, who only intervened on those infrequent occasions when 
the program malfunctioned or if any student failed to enter their name 
correctly at the start. 

Typically, students worked diUgently on-task during their computer study 
even though classes were continuously in session ia the two classrooms and 
there was a ftdr amount of activity (phone calls, counseling, eh ) in the 
counseling room. 

Description of the Experimental Treatment 

The treatment consisted of the following: 

• Initial practice using a disk that had no relevant text or 
content but required that each student learn 

(a) the saeen format 

(b) where the story would appear (lower part of saeen) 

(c) where questions and directions would appear (upper 
part of screen) 

(d) where feedback would appear (in a temporary memo 
window overwriting the story text) 

(e) where the scoring and cumtilative progress status 
would appear (presented as colored squares forming a 
line aaoss the middle of the saeen, together with a 
detective fee "earned" to that point) 

(f) how to use the "HINT* button to obtain additional 
information as needed (the HINT button was also 
automatically activated after a time-delay) 

(g) how to use the "DONE** button, signifying the end of 
study for a particular day (automatically marking the 
restart position for subsequent study periods) 

(h) how and when to use the n and b keys to advance to 
the Next page or fiack to the prior text page (used in 



passages where text material extended beyond the space 
available in the text window) 
(i) how and when to use the return key and spacebar to 
initiate the program and advance it from lesson to 
lesson 

(j) how to turn on ^the computer, haoidle the disks, and use 
the disk drive 

• Detective Story #l~7he Case of the Missing Brother 

• The story deals with the disappearance and finding of a 
younger brother while visiting Disneyland. There is a 
"reward" of $100 (20 questions). 

• The story begins with "easy" questions (Mark my 
brother's name.) and moves toward predse, literal 
questions (When did Jim first know that he was lost?). 

• D^^ective Story #2-The Case of the Sea Queen 

• The story deals with the disappearance and finding of a 
"ghost ship," th^ 5ea C!ueen. The story offers a reward of 
$120 (24 questions). 

• The story includes items in which the fact and the, 
question are. transformed, (He watched the....; What did 
he see?) and awareness of implicit facts (How many 
people were in the car?). 

• Detective Story #3-The Case of the Missing Bicycle 

• The story deals with the disappearance and finding of a 
missing bicycle. It offers a reward cf $115 (23 questions). 

• The story includes items requiring strict attention to the 
timing of events (John's bike was last seen at: 11: 20, 11: 
30, 11: 45, 11: 46) and moves toward inference based on 
clues (John's bike /fas probably at: the fire department, 
the police station, the library, his home) 



The Wordsworth program allowed each student to interrupt study at the end 
of each class period and resume study at the same point in the story on the 
next day. This feature made it possible for slower students to complete the 
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stories wib'iout retracing their steps through the lesson material. Mowing 
for absences, most students completed the training (practice) lesson and the 
three stories in less than four class periods. 

The computer program automatically kept track of student performance on 
each item for each detective story and loggec: elapsed time as well. At the 
condxision of the experimental study printouts were made of each students 
responses. These print outs detailed evcjy action taken (e.g., key presses) and 
every item "displayed (e.g., text highUghted) for all three stories. 



Results 

Not surprisingly, most of the students were glad to try the computer program. 
As mentioned previously, considerable attention had been paid to the way m 
which the students would interact with the Wprc^gworth software and the 
interface was simplified based on pUot testing. Accordingly, students did not 
find it frustrating to use the modified saeen display nor the mouse and 
simplified keyboard interface. 

They quickly learned and observed the rules for operatmg Wordsworth as 
presented in the spedaUy developed training program. The observed 
performance of the students on the main treatment (the three different 
stories) is summarized below in Table 15a. 

Table 15a. Study time for students exposed to the Wordsworth software. 

Studv Time (minutes) Story 1 

Low time 
High time 
Mediar 
Mean 



1 


2 


2 


15 


15 


11 


43 


53 


43 


26 


22 


20 


26.3 


27.1 


22 


25 


23* 


24 



N (combined groups) 25 

♦ Time data on two cases not saved on disk 
** Time data on one case not saved on disk 



It was dear that each of the stories required less than a half hour of study time 
for most students. About 75-miautes overall was the mean time for exposure 
to the treatment. This is an important consideration because it means the 
'combined amount of exposure to training is quite small given the desired 
outcome of improving students' comprehension ability. 
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It was noted that students were attentive to the screen. The deliberateness of 
their reading and searching for correct answers indicated a desire on their part 
to get high scores (earn high reward money on the computer screen) during 
their computer study. This was reflected in the attained scores as shown in 
Table 15b. 



Table 15b. Accuracy of study on each story for students exposed to the 
Wordsworth software. 

Accuracy Story 1 _1 _3 

MaxifiX'am score possible 100 120 115 

Low score 72 61 73 

High Score 100 118 115 

Median 95 109 107 

Mean 92.5 105.5 105.9 



The relationship was examined between the speed of completing the stories 
and the total score attained in the first story. Figure 3 shows a regression plot 
and associated statistics for all students exposed to the treatment Although it 
was not significant, there was a slight tendency for the students who took less 
time to have higher total story scores. 

Students were asked to evaluate the three stories after they completed all 
three. Most of them preferred the second story, which concerned the fmding 
of a "ghost ship." When asked why, they gave varied responses which 
generally suggested that it had a higher interest level. We speculate that the 
mysteries solved in stories one and three (a lost brother and a lost biq'de) 
may have been perceived by these secondary students as "a little young." 

Simple regression was used to examine the relationship between the total 
score earned in the three stories and the performance on the reading 
comprehension test after computer study. Figure 4 shows a strong positive 
relationship (p< .02) which indicates that the same students do well on 
Wordsworth activities and on the standardized comprehension subtest. 

Students participating in the study were drawn from grades 9, 10, 11, and 12. 
The question was considered as to whether, following Wordsworth study, the 
students were comprehending at, above, c? below grade level as measured by 
the Stanford Achievement Test, comprehension subtest. At the time of the 
posttest all but three of the students were still exhibiting a level of reading 
comprehension below their school-assigned grade levels (though somewhat 
higher than they had on the pretest). (See Figure 5.) The three who did score 



Figure 3. Relationship of study time on story 1 to attained total story scores 




Variable 
Nama 



Constant 
Time Story 1 



Der^ndant Variable: Total story sco 



Ccsffic'ont 



Std. Err. 
Estimate 



325.586 
-0.821 



19.112 
0.699 



t 

Statistic 



Prcb > t 



17.035 
-1.175 



0.000 
0.252 



SuTi of Oen. of Moan 

Source Squares Freedcm Squares P-Ratio Prob>P 

Modei 951.979 1 951.979 1.380 o7743 

Error 15868.021 23 689.914 



Total 16820.000 24 

Coefficient of Determination (R^2) 0.057 

Adjusted Coefficient (R'^2) 0.016 

Coefficient of Correlation (R) 0.238 

Standard Error of Estimate 26.266 

Durbin-Watson Statistic 1.550 
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Hgure.4. Relaficihship oi total story scores to scaled ppsttest scores 
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Dependent Variable: Scaled post 



Variable 
Name 


Cc^officier! 


Std. Err. 
Estimate 


t 

Statistic 


Prob > t 


Constant 


418.04? 


94.583 


4.423 


0.000 


Total story sco 


O.o32 


0.311 


2.736 


0.012 


Source 


Sum of Deg. of 
Squares Freeidom 


Mean 
Squares 


F-Ratio 


Prcb>F 


Model 


11842.025 1 


11842.025 


7.488 


0.012 


Error 


33210.534 21 


1581.456 







Total 45052.609 22 



Coefficient of Determination (R'^2) 
Adjusted Coefficient (R''2) 
Coefficient of Correlation (R) 
Standard Error of Estimate 
Durbln-Watson Statistic 




0.?,63 

0.228 

0.513 

39.768 

1.308 



Figtire 5. Relationship of assigned g^'ade levels to grade equivalent scores 
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Grade Level 



Variable 
Name 



Coefficient 



Dependent Variable: Grade Equrv 



Std. Err. 
Estimate 



t 

Statistic 



Prob > t 



Constant 
Grade Level 



7.814 
-0.051 



5.452 
0.508 



1.433 
-0.101 



0.165 
0.921 



Sum of Oeg. of Mean 

Source Squares Freedom Squares F-Ratio Prab>F 

Modei 0.095 1 0.095 0.010 0.921 

Error 214.820 23 9.340 



Total 214.914 24 

Coefficient of Ceterminauon (R^2) 0.000 

Adjusted Coefficient (R''2) -0.043 

Coefficient of Correlation (R) 0.021 

?* ndard Error of Estimate 3.056 

Ourbin-Watson Statistic 1.655 
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at grade equivalents higher than their school-assigned grades, tested at the 
-t high school level, shown as grade 13 in the figure. 

First a r alysis: Main Effect with Paired Control and Treatment Groups. 

Students were paired (matched according to the raw scores for the literal 
factor on the compxehension subtest) and randomly assigned to treatment 
and control groups. As shown in Table 16, based on raw scores for the 
matching aiterion, the treatment and control groups were very much alike 
(p< .84) at the time of the pretest. When comparisons were made --ween the 
treatment and control groups at the time of the posttest, again usi. ^ che raw 
scores on tlte literal factor, no significant differences were found (p< .34). This 
would seem to indicate that no main effect was obtained for the Wordsworth 
treatment insofar as the matching criterion was concerned. 

The Stanford 7 Plus Norms Book provides scaled scores only for the 
comprehension subtest (allowing direct comparison across alternate test 
forms) and does not separately report on the literal factor. The literal factor 
comprises 30 of the comprehension items, while the factor measured by the 
remaiiung 30 items taps inferential skills in reading. When the scaled 
comprehension scores were analyzed it was found that the pretest scores were 
still very similar for the two groups (p< .90). As shown in Table 17 the 
posttest scores were still not significantly different (p.< .15) though they were 
in favor of the treatment group. 

At least three alternative explanations exist for the seeming lack of significant 
main effect. First, the relatively short exposure to Wordsworth (with an 
average study time of less than 75 minutes) may have not been enough to 
bring about the desired change. 

Second, there may have been some "learning" on the part of control students 
who were present in the classrooms while the treatment group was studying 
the computer materials. This does not seem likely since the positioning of 
the computers was such that direct viewing by an observea- was only possible 
if he or she stood directly behind the student at the keyboard. Whenever this 
was observed by the staff it was terminated, but it cannot be stated with 
assurance that it did not take place when staff were not present. 

Third, the groups might not have been as well matched as indicated by the 
pretest scores. This would raise questions about the appropriateness of the 
group comparisons. Recall, for example, that in spite of random assignment 
five of the six "special day" students were in the treatment group. Their 
ability to benefit trom the Wordsworth experience may not have been as high 
as the learning disabled students who were classified as resource students. 
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Table 16. Pretest comparison of treatment and control groups usirtg 
raw scores on ttie "literal" matching criterion 

Paired Samples... 

Variable: C Literal Pre T1 Literal Pre 

Mean: 21 .000 20.667 
Std. Deviation: 3.766 3.869 
Paired Observations: 12 

t-statistic: 0.217 
Degrees of Freeddn: 1 1 
Significance: 0.832 



Table 17. Posttest comparison of treatment and control groups using 
scaled scores on the comprehension subtest 

Paired Samples... 

Variable: C Scaled post T Scaled post 

Mean: 650.385 665.077 

Std. Deviation: 26.588 42.970 

Paired Observations: 13 

t-statistic: -1.563 
Degrees of Freedom: 1 2 
Significance: 0.144 
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When the scaled scores for the special day and resource students are 
compared a picture emerges that shows significantly different populations. 
The special day students tended to be older (mean age = 17 years and 7 
months) than their counterpar'3 in the mains tre^'Jied, resource program, 
<mean age = 16 years and 9 months). As shown in Table 18 and Table 19, the 
special day class students were lower on the literal raw scores (p< .04) and on 
the scaled comprehension scores (p< .01). In other words, the cards were 
inadvertently "stacked" against a treatment effect in the group comparison. 

Second analysis: Main Effect with Students as Their Own Controls. 

After the posttests were admiiustered to both the treatment and control 
groups, the control group students were then:\selves exposed to the 
Wordsworth materials. Thus they became a second treatment set and 
another posttest was administered to them following their treatment 

When the scaled scores for the control group were compared grior to 
administering the Wordsworth treatment to the control group, no significant 
difference was found (p< 23). This was expected because we are merely 
comparing two pre-treatro/^t administrations of tixe comprehension 
criterion test. However, as shown in Table 20, when the scaled scores axe 
compared after the control group had received Wordsworth training, 
significant differences were noted (p< .01). 

As shown in Table 21, when comparisons were made between the pre- and 
posttest scaled scores for individuals receiving the first treatment only (the 
original treatment g^oup, which included five of the six special day students) 
there was an upward shift in performance on reading con^prehension, but 
not a significant difference (p< .15). 

Finally, a comparison was made of the pre and post comprehension scores for 
the combine d groups after each group had been exposed to Wordsworth 
materials. (See Table 22.) In this comparison a substantial difference was also 
noted (p< .01). 

These data show that a definite increase in comprehension skills was .oted 
for individuals exposed to the Wordsworth treatment, though the treatment 
does not seem to be as effective for lower ability students such as one might 
expect to find in the spedal day class category. 



Summary 

There was evidence that the Wordsworth software did help to raise the 
comprehension skills of the learning disabled students in secondary level 
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Table 18. Comparison of Spedai C^y and Resource students using 
rav.' "literal" scores used as matching criterion 

independent Samples... 

Variable: Literal SO Literal main pr 



Moaii: \ 13.333 21.800 

Std. Deviation: 3.077 3.458 
Observations: 6 20 



t-statistic: . -2.202 
Degrees. of Freedom: 24 
Significance: 0.038 



Tab*^ 19. Comparison of Special Day and Resource Students using 
scaled scores on the comprel-tension subtest 

Independent Samples... 

Variable: Ck)mpSD Comp main 

Mean: 32.000 44.750 

Std. Deviation: 5.933 8.724 
Observations: 5 24 



t-statistic: -3.368 
Degr&es of Freedom: 28 
Significance: 0.C02 
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Table 20. Gains in comprehension by individual students in the "contror 
group after exposure to the Wordsworth treatment 

Paired Samples... 

Variable: C Scaled, post 2 C Scaled post 1 

Mean: 689.900. 865.400 

Std. Deviation: 46.443 41.578 

Paired Observations: 1.0 

t-statistic: 3.346 
Degrees of Freedom: 9 
Signif' ance: 0.009 



Table 21. Gains in comprehension by individual students in the "treatment" 
group, including spedal day students 

Paired Samples... 

Variable: T Scaled post T Scaled pre 



Mean: 664.846 650.385 
J ^aviation: 43.066 26.588 
Paired Observations: 1 3 

t-statistic: 1.541 
Degrees, of Freedom: 1 2 
Significance: 0.149 
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Table 22. Gains in comprehension for all students after study of Wordsworth 



Paired Samples... 

Va riable: Comb Scaled pes Comb Scaled pre 

Mean: 675.870 652.435 

Std. Daviation: 45.253 25.536 

Paired Observations: 23 

t-statistic: 3.442 
Degrees of Freedom: 22 
Significance: 0.002 



resource programs. However, there is less reason to expect that the software 
would be of as much help to the more severely learning disabled who are in 
special day classes. 
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• 'Implications for Fuhire Research 
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Implications for Futixre Research 



Each phase of study in this multi-year project led to important findings that 
inform the use of technology fo/ special education students. 

In the early stages of the project our review of the literature led us to 
hypothesize a heirarchical model in which lower order skills were considered 
as preconditions for highc^ ovdf^x functioning. Our reasoning was that 
efficient reading performance for instructional or leisure purposes was 
dependent on efficient underlying cognitive skills. In accordance with the 
research literature on automatidty in reading, it was further reasoned that 
efficient cognitive processing was possible only if the perceptual enabling 
skills were "automatic" and did not require an inordinate cognitive-capacity 
demand. Similarly, the assumption was made that this "effortless," 
automatic perception would be difficult to attain if the image presented to the 
eyes was improperly fus(>d or if fatigue resulted from inefficient binocular 
accommodation. 

Resaarch within the CREATE project and a separate, but allied, vision skills 
project was designed to shed light on these relationships and to explore the 
potential for using computer technology to assist learning (reading) disabled 
youngsters fn bridging these heirachical skill levels. 

Implications concerning reading related visual, peceptual and cognitive 
processing skills 

Our findings suggest that substantial progress can be made in developing 
students higher order reading skills by addressing their lower order needs in 
cognitive processing skills, perceptual enabling sldlls, and (as shown in a 
related project) neuromuscular vision skills through carefully designed, special 
purpose computer software. 

The use of this type of special purpose software is seldom seen in the schools 
and generally is not being developed by software firms. This repres^^nts, in 
our judgement, a major gap in the tools that teachers should have to work 
with as they attempt to help reading disabled youngsters. 

The experimentdl phases of the CREATE study focused on detern^iining 
whether this kind of software ( TurboScan and Wordsworth) could have an 
impact and what features were important to have in software that is aimed at 
the learning disabled. The limited data from this study shows the potential 
for significant impact on elementary level students in the perceptual enabling skills 
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levd, but students without any decoding skills may not be benefited until these 
ai-e developed by the teacher. Similarly, a strong impact on secondary level 
students was shown at the cognitive processing level, though again not as much for 
students who are more severely reading disabled. 

In terms of software features, we were unable to experimentally identify any 
specific features that are essential to success of the software. We suspect that a 
critical mass of "good" inj'jructional design features is important but we can 
only suggest the appropriate principles of instructional design that have been 
known for some years in educs. ion and which guided our own development 
of the reseaidi tools. These principles mvolve dear directions, an engaging 
screen display that is free of irrelevant distractions, provision for feedback or 
knowledge of results to the student, and challenging, relevant content set at 
an appropriate level of diffioxlty for the learner. 

Xmplicdtions concerning software/hardware design 

In ta-ms of student/machine interface design, we able able to demonstrate 
convincingly that "less is better." That is, in terms of software display the less 
the learner has to know and remember to "operate" the sf .vare the better. 
In addition, the motor skills and manipulative acts required to enter 
responses or cou. 'ol the stimulus on the screen must be kept simple. The 
task set before the designer of configurations of hardware/ software for the 
disabled is to ensure that the required student/machine interaction does not 
become a source of frustration or distraction. 

For example, in our TurboScan study we learned that "press any key" entry is 
successful, even when timed responses mtist be made. In our Wordsworth 
study we learned that mouse input is easier than joystick input or keyboard input 
when care must be taken to direct the cursor in order to activate specific areas on 
the sCTeen. 

Finally, although it was not specifically included in our experimentation, our 
research leads us to believe that the computer may be the ideal tool for ass^sing 
students' existing skill levels in reading and diagnosing deficit areas in underlying skills 
We refer here to the heirarchical skills presented in the CREATE model 
presented in an early chapter of this report. We believe that the development 
of such diagnostic computer tools should be one research and development 
area that receives priority funding by OSEP. 

Implications for the evaluation, selection and integration of software 

Currently, the pattern of software use in the schools seems to emphasize 
soil-ware that has been "proven"' either by frequent school purchase or because 
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of word of mouth testimony. Subscribed "reviews" of new software by 
software review firms or in articles in magazines and newsletters also lesds to 
some purchases as does "the reputation" of specific software development 
firms. Our shidy did not lead us to believe that.much evaluation and 
selection is approached objectively and formally by the schools. This may be 
understandable in terms of the relative cost of the software in contrast to the 
time it takes to do a careful evaluation. However, this argument begs the 
question of what schools are all about - the presentation of learning 
opportunities that are ap p r o p riate for the different students in the schools. 

We believe that efficient evaluation and selection procedures are possible and 
should be put in place in each school district They should help teachers 
identify the special purpose software that handicapped students really need rather 
than only adopt existing software that deals with academic performance skills, 
and which is primarily targeted at the mainstream populations of students. 
The prototjrpe evaluation and selection form developed in CREATE (shown 
in Appendix B) can serve as a start in that direction. 

When standard software is going to be used with the learning disabled (and 
with other students as well) it makes sense to integrate the computer experience 
xvith the regular classroom program rather than treat it as a separate, imrelated 
"reward." Toward this end, more adaptive workbooks should be developed along 
the lines of the Factory and Reader Rabbit Workbooks that v/ere jointly 
developed and field tested with educators during the CREATE project. 
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Appendix A 
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I 

SCOPE AND SEQUENCE 

j =na^Hnr; .qkill A fPisk A: scrgQns 1-56\ 

To identify tne attnbutes of symbols and characters that differentiate then from each 
other, and to make rapid discriminations between shapes that are similar. 

Lavei 1 . Reccgnize and differentiate forms based on gross 
characteristics. 

Concepts: I jtinguish straight, curved; open, dosed: large, small; 
diagonal, vertical, horizontal. 

Example: IC; o c; O o; / 1 - 

Level 2. Recognize and differentiate forms based on directionality and 'terality. 
Concepts: Distinguish top, bottom; left, right; up, down. 

Example: f "^j f J 

Level S . D'-criminate features based on component analysis. 

Concepts: Less than, more than; part of, all of; same, opposite. 

Example: n, m; F E; R P; p b 
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^r^r^Wnq Skill B fPigk 5: scrggns 1»63^ 

To distinguish partituiar symbols or characters, and pattP/ns of synbois or characters, 
wnen presented ih a baci<ground field of distractore. 

Level 1 ■ Perceive the form and position of k symbol or character in order to 
compare it to a specific target. 

Concept: Relative shape and position on a horizont?' line. 
Example: 

Leve' 2 . Perceive the form and position of a character in order to compare it to a 
target character in a particular orientation. 

Concept: Mai: n relative orientation of significant feature. 

Example: b bpb ddp pdb dbp 

Level 3. Perceive a set of characters as a unit within the field. 

Concept: Maintain form constancy regardlass of field position. 

Example: ch change reach cheap catch 
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3 



0 g'^sbiipg Skill C fpisk C1 • scregng i.TQ- Q'sk C2: sceens 1-20^ 

To distinguish selected features of words that ccntnbute to sight recognition (initiai. 
character, word shape, and word length). 

Lgyg! 1 : Recognize initial and ending letter combinations. 

Concept: Quickly classify wonjs by their beginnings. 

Example: Beginning wh~Where were those when I wanted them? 

Level 2: Recognize the ascender and descender patterns of particular words. 

Concept: Quickly classify words by the presence or absence of ascenders 
' and descenders in character sets. 

Example: Ending ght 

I was right to hold tight 

i 

Level 3 : Distinguish the relative lengths of word3. 

,oncepf Differentiate words based on perceived length. 
Example: Find three letter words 
You and I went to the store. 
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^nsbiinc "gkiH D ( Disk D1: scresns "•-•S'^: Disk 02: scrggns *"3^\ 

To recognize panicular combinations of characters as being meaningful. 

Level 1 : Distinguish sets of characters that constitute a target word 
when presented within a field of words. 

Concept: Identify specific words by sight. 

Example: Rnd through 

Go through the door and then through the hallway. 
Be sure to call when you are through. 

Level 2 : Distinguish acceptable and unacceptable letter patterns within a field. 

Concept: Proofread for errors. 

Example: He siad that was OK. dut she said it was not. 
Level 3 : Apply visual memory to silent and oral reading. 

Concept: Combino word beginnings and endings appropriately. 

Example: Rnd and say the words that start with ex. 
__periment p«q plain tra 
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ffi Skill Categories Used in TurfaoSean study 3 



Skill Category 


Instruction line 


Sample from Screen 


Initial Letter 


"Rnd each word that 
sians witn o. 


go he do by if of oh It be 
had but how did big day old 


Initial letter 
combination 


"Find each word that 
starts with st" 


StOD tum trPA ^hirt ct£)\/ Hatf 

wvw^ IWIIS UQC7 OvCty wiOY 

trees still short sho^^ stand 


llength 


"Rnd words as 
long as the dashes" 


sun so song sea sing 

cut city cat call care 


Length and Initial 


"Rnd words like this." 


^ so she safe said saw 

salt sad school setds 


Length and initial 
letter combination 


"Rnd words like this." 


St stand stay strong story 

stop study str?ng step 


Shape 


"Rnd words 
with this shape." 


to at be go am by 
on he my oh of do 


Shape 2nd 
initial letts; 


"Rnd words like this." 


bl^ bam bark bear bird 
f^i fish fine fast fonn 


Comprehension 


"Find all th« Animalc " 
1 II lu cut 11 19 oi nil laid. 


door bin hnr^p t^^hlA nin 






dog was chickens chair 


Find and Say 


"Rnd and say she." 


She said that is one can. Is this 
what she said? She said tha: 
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SCOPE AND SEQUENCE 



Cogititive Skills 

to read with comprehension. ' — 

Level L Attend to facts an4 details in narrative ext 

Concept: Identify specific infestation in response to a question. 

Example: Joe went shopping at. the supermarket He had six 
dollars in his pocket He wantevd to buy soir^ inilk, bacon, a 
toothbrush, crackers and gum. He would ou\y buy the gum if he 
had at least 45 cents left after he bought the other things. The 
milk was $1.69, so he bought one bottle The bacon was $2.15. 
H saw a sale sign in the toothbrush section and got one for only 
59 cents. There were lots of crackers to choose from, and he 
chose some cheecs aackers for $1.19. The total came to $5.62. 

How many items did Joe buy at the store? 



Level 2, Maintain a logical order in a sequence of events in narrative 
text 

Concept: Identify relationships between a series of activities. 

Txample: The peace conference was very successful. A week 
before the peace confer^rnce government officials had met to 
discuss matters diat most concerned the leaders of their 
countries. A list of topics was prepared and sent to all the leaders 
who would be attending. When the conference began each 
leader already knew what the other leaders would be discussing, 
TO there were no surprises. After each leader spoke there was 
tima for responses by the other leaders. The last thing Ihey did 
was to sign the agreement thut was reached. 

How did the leaders know what would be discussed at the 
conference? 
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2 

Level 3. Draw inferences b?ised on the content aitd context of 
information. 

Concept Apply reasoning to form conclusions based on facts 
supplied. 

Example: Tears came to Mary Ann's eyes. The house looked 
smaller than she lamembered* it She had been away so long she 
had forgotten how the place looked. As she glanced around she 
— remembered little things. There was the perch her mother and 
dad used to sit on in the evening. There was 'che back yard 
where she played with the puppies. The swing that hung from 
the apple tree was gone now, but she comd remember how 
much fun it was to be pushed by her older brother. Maiy Ann 
wasn't crying because she was sad, but because conung back was 
so important to her. 

What place did Mary Ann came back too?' 

L evel 4. Identify the main idea in nairative text 

Concept: Combine narrative information to form higher level 
abstractions. 

Example: Young people today are able to do more things, go 
more places, and have more fun than they did when I was 
growing up. fait of the reason is that so many of them have cars 
to gt around in. But I read the other, day that they also have the 
mostacddents with cars. If they knew how one accident could 
change their lives forever, I wonder if it would make them more 
careful. I hope so, because one quick mrtake is all it takes to 
hurt someone they love or even themselves. 

What is the main idea in this paragraph? 
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CSSATS : The Center for Fisearch and Evaluation in the Applica:.cn cf Tecr:nolcgy to Educat.cn 



Software Evaluation and Ssloction Form 



Field Test: October 1985 




A^1ER1CAN INSTITUTES FOR .lESEARCH 
fN THE BEHAVIORAL SCIENCES 
P.O. Box 11-13: 1791 Afastradero Rd., 
Paio Alto. CA 94302 • 
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American Institutes 
for Research (AIR) 



Software Evaluation and Selection Form 

The Purpose of this Form 

S'^ucatofs everywhere are intriged by the prospect of using computer technology in educationally 
scunc ways to .mprove the teacmng and learning process. A key to doing so. however, is ensunng 
that the software that is selected and used is of high quality and serves a specific purpose that is 
consistent with the instoictional mission of the schools. This document will help in locating, 
selecting, and describing the software that scfwols may want to acuire and use. Moreover, it is 
ces;gne<3 to accommodate the instmaional needs of special education students by cifferentiating 
between academic performance skills, cognitive processing skills, and the perceptual enabling 
Skills essential to the development of understanding and appropriate behaviors by students. 
There are four s ections to the iatm. Oirecttons for completing them follow. 

I. Program-Source Data 

Who should complete this section: TWs information can be filled out by 
clerical staff, classroom aides, or volunteers before the program is reviewed by facufty. 

How to complete this section: RI in the infoNTOtion as supplied with the software package. 
This infonnation is usually found in the first few pages of the documentation. 



11. Program Evaluation 

Who should complete this section: This section might be filled out by a designated 
reviewer representing the district o' severaldistricts. Typically, this would be a resource specialist. 

How to complete this section: Clrele a rating score of 0-4 for each item. If information 
or materiate are not indudev but are applicable, drrfe a rating score of 0. If infonration or materials are 
not applicable, circle the X. To obtain an average rating score for each section, divide th6 sum of the 
rating scores by the total number of Hems rated 0-4. For example: 



III. Appropriate Applications 

Who should complete this section: Both regular and special education teachers 
should complete this section. 

How to complete thiC section: check off each sWH that is erther directly or indirectly 
taught by the program. 

IV. Summary of Evaluation and Selection Data 

This is a simple transfer of key information for ready accoss. 




Add number of items 
that were rated 0-4: 



0 



' (a) 3 
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I Proqram-Source Data completed by:. 

'• 5' Organization:-, 

A. Product and Source Description: 



1 . Program Name: . 

2. Distributor Name: 

3. Address: . 



4. Telephone: 

5. Single Program or Part of a Series . 

6. Cost: — — : " 

7. Terms for Updated Versions: ^ 

8. Back-up Policy: How many copies includefj'^ 

Costs per copy 
Temis for multiple copies 
Hard disi; version available? 



B. Hardware Requirements 



1 . Program runs on these microcomputers: 



2, Memory required: 

3, Equipment requirements: 

4, Optional Devices: 
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C. Educational Objective: 



1 . Instructional Purpose: 
n Regular Instruaion . 

[j Rsn^eciation 

O Enrichnient 



2'. Suggested Grade/Ability levels: 

3. Presentation mode: 

n Drill and practice 
n Educational game 
n Problem Solving 
□ Simulation 
Q Tutorial 



Q Classroom management 
n Information retrieval 
Gl Authoring System 
[H Game (for fun only) 
[n Testing 



4. Specially relevant to special education students: 



D. Summary of Program Content: 



f ERIC 
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II. Program Evaluation 

A. Instructional Design Features: 



Completed by:. 
Organization:. 



Instructional Strategy: 


Not 

Aooiicable 


NO 

Induded 


Poor Excellent 


Lsarner "a infcmoc cf oc^activo 


X 


0 12 3 4 


Sranc^ei '.s acjust :a laamer's aoiiiry 'ev«i 


X 


[ 0 1 1 2 3 4 


Gives ~:nts ;o encourage carrec answer 


X 0 


12 3 4 


Instrucar can adaot'moflify program 


X 0 


12 3 4 


1 En^oioys a sound theoretical bas« X 


0 


12 3 4 



Sum of rating scores: 



Numbor of items 
that woro rated 0-4: 



= Overall Average: 







Not 


Ratinq Scale 


Learner Control: 


Applicable 


Included 


Poor 




Excellent 


Can control frame speed/movement 


X 


0 




2 


3 


4 


Can select activity 


X 


0 




2 


3 


4 


Can repeat any activity 


X 


0 




2 


3 


4 


Can get dues on- line 


X 


0 




2 


3 


4 


Can get help on-line, at alS times 


X 


0 




2 


3 


4 


Can exit at anytime 


X 


0 




2 


3 


4 



Sum of rating scores: 

Number of itema 
that were ratod 0-4: 



= Overall Average: 



Feedback/Reinforcement: 


Nat 
Applicable 


Not 
Included 


Ratirx] Scale 


Poor Excellent 


Positive reinforcement is immedisto ' 


X 


0 


12 3 4 


Incorrect responses are explained 


X 


0 


12 3 4 


RameJial loops are built in 


X 


0 


12 3 4 


Rewarded for correct responses 


X 


0 


12 3 4 


Daily assessment d performance 


X 


0 


12 3 4 


Cu' '' tive assessment of performance 


X 


0 


12 3 4 



Sum of rating scores: 



Number of items 
that were rated 0-4; 



= Overall Average: 
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L B. Quality of Individual Components: 



Documentation: 


Not 1 Not 
Aootica'' ^( Included 




Poor Excellent 


Useful taacn^r's guictt mfcrmaton 




0 


12 3 4 


CJear infcrmation for learners | x 


0 


12 3 4 


Cear ooerarirg msn^cscns 


X 


0 12 3 4 


CJear trtstnj^onai cpjectives 


X 


0 


12 3 4 


Clear instructions on how to mocify 
program features/ccnttnt 


X 


_0 


12 3 4 


Ust of prerequisite skills ' 


X 


0 


12 3 4 


. Sum of rating sooree: 1 

Number of items jT' 
that were rated 0-4: 


-« — . Overall Average: 


1 "1^ 


Supplementary Materials: 


Not 
Applicabis 


. Not 
Included 


Hatina Scale 


Poor Excellent 


Provides learner woricjheeta 


X 


0 


12 3 4 


Provides instructor reasrd sheets 


X 


0 


12 3 4 


Provides learner record sheets 


X 


0 


12 3 4 


Provides test matefials 


X 


0 


12 3 4 


Provides follow-up activites 


X 


0 


12 3 4 


Sum of rating scores: 

Add number of items 
that were rated 0-4: 




— Overall Average: 





Program operation and 
Presentation: 


Not 
Applicable 


Not 
Included 


Ratini^ Scale 


Poor Excellent 


Grammar/punctuation correct 


X 


0 


'12 3 4 


Safeguarded against 'crashes* 


X 


0 


12 3 4 


Text on the screen Is readable 


X 


0 


12 3 4 


Graphics have instmctional intent 


X 


0 


12 3 4 


Color can be, controlled 


X 


0 


12 3 4 


Sound can be toggled on and c^^ 


X 


0 


12 3 4 


Menu driven 


X 


0 


12 3 4 


Help available on-line 


X 


0 


12 3 4 


Sum of rating scores: 

Number of items 
that were rated 04: 


I 


— = Overall Average: 



o 
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111. Appropriate Applications ^q';^^^^^^^^ 



A. Developing Enabling Skills Involving Perception 



1« Discrimination 
n features 
Q patterns 

Jl. Attention 
n| concentration 

n alertness 
O spari 

3. Classification 
Q grouping 

Q comparing 

4. Cue Idantificatlon 
Q searching 

□ ditrei^ntiation 

5. Cordination and 

Integration 

n visual/motor 
rj auditory/motor 
n visual/auditory 



6. Tracking 

□ left to right 
Q top to bottom 

7. Part-whole Relationships 
JT] verbal 

□ graphic 

8. Spatial Orientation 
n laterality 

n figure-ground 

9. Decoding of Letter Shapes 
Q typed or printed 

n reversals/Inversions 

10. Decoding of Words 
Q letter combinations 

Q word length 

n shapes (ascenders and 
descenders) 
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B. Developing Processing Skills Involving Cognition: 



1. Anaiytiu Techniques 


3. Recall of Information 


Q in reading 


n chunking & grouping 


riin math 


n mnornonic techniques 


n in science • 


n rehearsal 


ri social studies 




2. Understanding and Meaning 


4. General Purpose Thinking 


n vocabulary and definitions 


Q reasoning 


n use d context 


n problem solvinw^ 


n comprehension 


n study strategies 
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C De v&ioping Performance Skills Invnlving Subject Matter 



1. Reading 
Q speed 

n ^hole word recognition 
n phonics and blends 
Q comprehension 

□ other^ 

2. Writing and Con:;munication^ 
Q spelling 

fl punctuation 
Q capitalization 
Q word usage 

parts of sj^eech 
ri organization of ideas 
Q syiUax 
Q proofreading 

□ other ^ 

3. Math 

Q acquisition of facts 
n number relations 
Q math operations 
Q problem solving 

□ logic 

□ graphing 

□ other 



4. Science 

Q] acquisition of facts 

Q observing and measuring 

Q interpretation of data 

Q fomiulating generalizations 

problem solving 
□ other^ 



5. Social Studies 
Q acquisition of facts 

□ interpretation of data 

n identification cf concepts & issi^es 
Q evaluation of evidence 
formulating genei^iiizations 

□ other 



6. 



Vocatlvnal/Buslness 
Q career awareness 

Q work readiness 

n technical skills 

Q other; 



?• Foreign Languages 
Q vocabulary 

Q rules and patterns 
Q comprehension 
□ other 



8. Arts 

Q] perception of visual properties 
ri analysis and interpretation 
Q expressive techniques 
P] aeativity of c ^sign and form 

□ other 

9. Computer Literacy 

programming 

n graphics 

n keyboarding skills 

□ other 



10. Personal 3nd Social Skills 

□ 

11. other Subject Matter 
□ _ 
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IV. Summary of Evaluation and Selection Data 



Proqram'Souree Data 



A. Frccram Name: 



S. Microccmpi-ter(s) and Memory Required: 



C. Educational Objective: 



D. Summary of Program Content:. 



Program Evaluation 



instructional Design Features: Rating Score Average 

Instnjctionai Strategy 
Learner Control 
Feedbaci</Reinfofcement 



Quality of Individual Components: Rating Score Average 

Documental *n 

Supplementary Materials 
Program Operation 



and Presentation 



sum of rating 
score averages: 



□ 



6 



= Overall Program Average: 
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Appropriate Applications 



This program is best used to teach: 



Enabling Skills 



□ 



Processing Skills 



□ 



Performance Skills 



□ 
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